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ABSTRACT 

The mission of the Far west Consortium for DDCE 
training is to design, develop, implement, and evaluate a 
ccHnprehensive and transpoirtable training program to meet the demand 
for trained personnel in educational development, dissemination, and 
evaluation (DOSE) . The program provides both preservice training and 
continuing education to upgrade those already employed in the field 
of educational research and developnent. The content of training has 
been derived by assessing personnel training needs and conducting 
task .analyses. .Training is organized around 8 functional competence 
areas: (1) planning and design; (2) collection and organization of 
information and data; (3) communication skills; (4) developmental 
engineering; (5) evaluation; (6) problem analysis and definition; (7) 
dissemination and marketing; and (8) management, in addition to the 
main report, a volume of appendices presenting such items as 
descriptions of competence areas, resximes of professional personnel, 
and trainee information and 2 attachments presenting the approach to 
competence assessment jmd the program of instructicai are also 
included. (Author/ES) 
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ABSTRACT . 

The mission of the Far West Consortium for DD&E training is to 
design, develop, implement, and evaluate a comprehensive and transportable 
training program to meet the demand for trained personnel in educational 
development, dissemination, and evaluation (DD&E). The program provides 
botn prerservice training and continuing education to upgrade those 
already employed in tfie field of educational R&D. It is designed to 
operate within the context of the jobs for which it is preparing trainees. 
It has been deliberately designed to be flexible and adaptable so- that it 
can be implemented in a variety of settings. The Consortium, which has 
been in existence for almost two years, will continue as a training devel- 
opment consortium until Fall 1974, at which time the training program and 
materials developed will be ready for dissemination and utilization. 

The content of training has been derived y assessing personnel 
training needs and conducting task analyses. The design of training 
follows a functional-context approach. Formal training at colleges or 
universities in conjunction with Engineered Internships at educational 
R&D agencies will lead to the MA degree. The continuing education program 
for practicing DD&E personnel is provided for professionals at the entry 
level. Work was also begun on development of a parallel paraprofessional 
program leading to an AA degree. Because of resource limitations and 
tne current low demand for paraprofessi»nals in DD&E, work has been suspended 
on this part of the project. 
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Training is organized around eight functional competence areas: 
(1) pi an ling and design, (2) collection and organization of information 
and data. (3) communication skills, (4) developmental engineering, (5) 
evaluation, (6) problem analysis and definition, (7) dissemination and 
marketing, and (8) management. Competence assessment instruments pertinent 
to the eight functional competence areas ars also being developed and 
validated. 
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PREFACE 

This document provides both a progress report and a projection of 
future accomplishments of the Far West Consortium. While using this 
report, the reader should also consult the design document submitted to 
the Research Training Branch at the end of 1970: Hood, Banathy, Ward, 
ct-aK, Design of a Functional Competence Training Program for Development , 
Dissemina tion, and Evaluation Personnel at Professional and Paraprofessional 
Levels in Education . 

This report is organized in tvjo volumes. Volume One' contains two 
main sections: Summary of Consortium Accomplishments 1972, and Work 
Plans for January 1 - May 31\ 1973. Volume Two includes the appendices 
which supplement Volume One. 

All Consortium members have contributed to this report, but we 
would like to acknowledge our special appreciation of Dr. George 
Hallowitz of California State University at San Francisco, James Dunn 
of AIR, and Robert Bennett of San Mateo Community College District for 
serving as members of the Consortium Planning Conimittee. 

We would like to acknowledge the help of Joe Ward, Wayne Rosenoff, 
Freeman Elzey, and John Hourigan of the If\structional and Training 
Systems Program of the Far West Laboratory. Special thanks to Nelly 
Sampson, Anne Macahilig, and Diana Studebaker for their editorial, 
coordination, and supervisory efforts in the preparation of the report. 
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THE FAR WEST CONSORTIUM: AN OVERVIEW 

The mission of the Far West Consortium for DD&E Training^ is to 
design, develop, test, and evaluate a comprehensive and transportable 
training program to meet the demand for trained personnel in educational 
development, dissemination, and evaluation (DD&E). The program provides 
pre-service training ^nd in-service training (continuing education) to 
upgrade the skills of those already employed in educational R&D. It 
is designed to provide learning opportunities directly related to the 
trainee's present or projected vocation. Since the program is both 
flexible and adaptable, it can be implemented in a variety of settings. 

The Consortium, formed almost two years ago, plans to continue as a 
training development group at least until Fall 1974. At that time, the 
training program and materials described in this plan will be developed 
and ready for national dissemination and utilization. 

The program is producing and field testing a series of training 
modules for each of eight educational DD&E competence areas: (1) Planning 
ri uesign; (2) Information/Data Collection & Organization; (3) Communication 
Skills; (4) Developmental Engineering; (5) Evaluation; (6) Analysis & 
npfinition: (7) Dissemination & Harketinq; and (8) Management. Modules 
developed in thestj competence areas can be used flexibly to meet specific 
individual training needs. 

To avoid duplication of effort, already developed instructional 

T . — 

A list of Consortium members will be found in Appendix A. 
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materials that fit into the DD&E competence scheme will be selected iFrom 
other sources and included in the instructional program. An example 
is the Educational Information Consultant Instructional System^. 

In place of traditional lectures and class meetings, the trainee in 
this program works with the learning episodes presented In the modules, 
performs activities and exercises, uses resources available in a learning 
laboratory, and works with other trainees. An Instructional Resource 
Manager functions as a guide and resource person rather th3n as a' "teacher." 

Whether pre-service or in-service, each trainee participates in an 
Engineered Internship. This internship, integrated closely with the 
"academic" training, provides the trainee with an opportunity to apply 
the skills he/she is acquiring in an actual DD&E work situation. The 
internship program is developed jointly by the trainee, the Instructional 
Resource Manager, and the trainee's work supervisor. All three meet 
periodically to plan and evaluate the trainee's progress. In the pre- 
service program, this joint planning and evaluation effort requires the 
academic institution to join forces with an R&D agency, bridging the gap 
between the academic and work domains. In the continuing education program, 
an R&D agency can sponsor its own program or cooperate with an academic 
institution. 

• The Competence Assessment System includes instruments, criterion- 

i referenced to the competence areas, to determine the trainee's competence 

An instructional program developed by Far West Laboratory consisting of 
seven modules that develop competence in disseminating educational infor- 
mation, see Appendix L. 

ERIC 
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both before and after traininn tu r 

training. The Competence Assessment System also 
provides means for certifvin^ ^ ^ . 

certifying competence without formal training. 

consortia activities began in ,970. Studies of national m 
personnel requirements were ana„.ed3 and an empiricaiiy derived need- 
assessment base ascertained „,at types and quantities of pe.onne, would 

2;TTJ" " ''''' " "^'^ °' - - --al 

-a - the Far West Consortium's geographic region in particular m 

addition, studies were made of the structure of. DD.E Jobs to assure that 
the training would be rplpv-^rn- n ■ 

relevant. Design of the DD&E training program was 
completed late in 1970^. 

The consortium began its operations in February 1971. with develop- 
-tal „„r. to implement the design. A test site for the pre-servi-ce 
program was established at the r;,nf • . 

(CSUSF, where an m degree program was initiated with concentratio 
educational DDSE. As training materials were developed, they were 
'n-duced and prototype ^.ted during Fall 1.1 and Spring 1.7. .ring 
fan 1972. testing of newly developed materials continued and prototype 
testing of the Engineered Internship was begun. 

Wo^k was also begun on the development of a parallel paraprofessional 
program in cooperation with CaKada Co«nity c liege. However, the cost 
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of fully developing high-quality, v^alidated, and transportable materials 
for both the entry-professional Itiel and the paraprofessional level 
programs now appears to exceed immediately forseeable resources. Moreover, 
the current demand for paraprofessionals in educational DD&E is not great. 
Therefore, the paraprofessional program has been =:ssigned a much lower 
priority. Its further development will be accomplished only if the 
requisite paraprofessional instructional resources are direct by-products 
of the development of entry-professional -.naterials and if the full 
development of the entry-professional program will in no way be jeopardized. 
Furthermore, in order to concentrate on the entry-professional program, 
tne design of an advanced level prograsn, initiated in 1972, will also be 
discontinued. 



Section One: SUMMARY OF 1972 CONSORTIUM ACCOMPLS I HMENTS 



I. Analysis of Projected Impact j 

II. 1972 Accomplishments 
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I. AN ANALYSIS OF PROJECTED IMPACT 
This analysis deals witn the projected impact of the Functional 
Competence Training Program on education and R&D training. 

A. General Levels of Impact 
We will discuss the direct and derived effects of the Functional 
Competence Training Program on immediate, intermediate, and ultimate 
target groups. 

Immediate target group: direct effect . Our knowledge of the extent 
to which the DD&E program has effected the trainees' mastery of R&D 
skills will depend on the validity of measuranent attained through use 
of the Competence Assessment System. A description of this system and 
sample competence assessment instruments are inclu'ded in Attachment One. 

Immediate target group: derived effect . The degree to which the 
immediate target group has acquired competence in DD&E skills will be 
determined not only by assessing the direct effect of the training as 
evidenced by the trainee's gains in knowledge and skills, but also by 
the derived effect v/hich is manifested in the trainee's application of 
learned competencies on the job. The Competence Assessment System will 
provide methods for making these assessments. 

Intermediate target group: derived effect . The intermediate target 
will be the agencies employing personnel trained by the program. It is 
anticipated that the availability of personnel v/ith certified competence 
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will materially increase the efficiency of hiring and placement processes. 
An employee at the entry-professional level who has had pre-service 
training can be put to work almost immediately without spending much 
time on orientation and on-the-job training. This speed-up should 
result in more efficient use or the agency's time and money. Availibility 
of a program for continuing DD&E education should also make it possible 
for personnel already employed to acquire new skills or improve existing 
ones thus increasing their value to the agericy and helping them fulfill 
career aspirations. 

Moreover, for those agencies providing Internships for DD&E trainees 
there s:iay be more subtle effects. It is possible for a trainee's presence 
in an agency to have effects which radiate out from his training situation 
through the ranks of the personnel who must analyze and re-examine their 
own tasks and organizational requirements in or'** work out suitable 
internship experiences for the trainee. This would be especially true 
for the trainee's work supervisor. More specifically, the work supervisor 
v;ill have to consider organizational priorities and requirements, and 
analyze the various tasks his/her group is involved with in order to 
identify those tasks that are best suited to the trainee's application 
of competencies acquired in the training program. Furthermore, the 
supervisor is asked to monitor the trainee^s application performance 
systematically on a regular basis. Although these requirements involve 
no more than good supervisory practice in work planning, supervision, 
instruction, and performance review, they are not commonly encountered 



9 



in many R&D agencies. 

Another intermediate target group includes those institutions 
engaged in training educational R&D personnel. The program offers them 
an organized and validated set of materials and procedures for the 
training of R&D personnel, serving existing and future training requirements 
that face these institutions of higher learning. 

Ultimate ta rget group . For any project that trains personnel who 
will be involved in educatirnal development, the ultimate target group 
includes the students or school personnel who will be using the R&D 
products that emanate from the development agencies. It is expected that 
increased efficiency in training DD&E personnel will rosult not only in an 
increased capability in development^ dissemination, and evaluation, but 
in more effective educational solutions and higher quality educational 
products . 

B. Impact of R&D Training 
1. Need for and Use of Training by Target Groups 

Two major sources suggest a heed for the training programs produced 
by the Far West Consortium, the first is the changing attitude toward 
support of educational R&D enterprises as pointed up by the formation of 
NIE. This new agency reflects a general political recognition that 
improvements in educational practice depend on a concerted and consistent 
effort extending over long time periods, rather than "crisis" or "crash" 
programs. This support should provide encouragement for R&D agencies, 
and give the development, dissemination, and evaluation aspects of R&D 
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greater professional appeal. Many of those now involved in "pure" 
research in universities may be attracted to these areas in R&D centers 
and educational laboratories (Schutz, 1972). 

The second indication of the potential usefulness of Consortium 
programs stems from the growing need to decrease the cost of higher 
education for both learner and institution. At present, these costs 
seriously tax the universities' capabilities; however, there are a 
number of new and promising design concepts and training practices 
emerging which may help solve the problem. Many of these new character- 
istics are incorporated into the design of the Functional Competence 
Training Program, including: 

a. Recognition that the direction of the individual learner*s 
activity should be determined by personally relevant goals. This 
notion implies that any and all available resources may be legitimately 
tapped, no matter where they are located. Modern communication techniques 
offer the learner a multitude of resources; instructors, training material 
laboratories, etc., are among the many he draws upon. The notion 

of "universities without walls" reflects and reinforces this concept. 

b. The development of validated, transportable instructional systems 
will offer a convenient, self-contained curriculum suitable to a wide 
range of learners. These systems, such as those being developed by the 
Far West Consortium, are modular in format, thus flexible. They are 
self-paced, individualized, and accompanied by appropriate assessment 
devices. They may well supply more cost-effective instruction than the 
traditional class instruction mode. 



c. A third important concept is the learner performance-oriented 
approach. Learners are beginning to ask their training institutions 
just what they will be learning, what they will be able to do when they 
have completed the learning experience, and how the learning experience 
relates to their personal goals and needs. The DD&E programs, specifically 
designed to answer such questions, should prove more attractive than 

most traditional curricular offerings. 

d. Other characteristics of the DD&E program that should enhance 
utilization include: organization of "banks" of modules and instructional 
resources; structuring instruction into preparation, intensive learning, 
and application phases; and availability of alternative delivery modes 
such as pre-service, continuing education, institute, and workshop forms. 
These characteristics have great potential to influence other training 
models in the educational field. 

More important than the single items mentioned above is the combined 
effGct emerging as components with the characteristics describe.d above 
are brought together within the systematic framework of an overall training 
desiqn and are implemented as a full-scale training oroorar^j. 
2. Projection of Use 

A projection of the potential use of Far West Consortium materials 
is offered here in terms of the kinds of products under development. 

Training niaterial s. The potential use of training materials is 
projected at three levels: 

a. The comprehensive .functional Competence Training Program offers 
the highest level of use. This program is expected to be used in both 
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pre-service training and continuing education. With aggressive "marketing 
somewhere betv/een 10 and 20 state colleges, universities, and private 
colleges can be expected to adopt the comprehensive pre-service entry- 
professional (MA) system during the first two years following release 
in 1974, with a potential user group of 100 to 250 students. Similarly, 
extension programs and continuing education programs could serve an 
additional 100 to 200 students employed in R&D agencies. The training 
content of the Functional Competence Training Program can je presented 
in other delivery forms, such as workshops, summer institutes, seminars, 
and ore- or post-conference sessions. 

b. Training modules are grouped by design into competence areas, cal 
series, and can be combined into a variety of "packaged" module clusters . 
Forir.ation of module clusters within series and across series, tailored 

to the particular needs of the trainee, provides a wider area of 
application for these materials. Potential users of such "packaged" 
programs will include (a) college and university schools of education, 
(b) private and public educational R&D agencies, and (c) state, regional, 
and federal educational agencies interested in staff development. The 
number of potential users for these packaged module clusters (used in 
workshops, institutes, individually, etc.) may range from 100 to 300 in 
the first two years of dissemination. 

c. The smallest unit of the DD&E training materials is the module . 
We plan to produce more than 30 modules at the entry professional level. 
Purchasers of modules may include: (a) libraries at most schools of 
education and at federal, state, and intermediate educational agencies; 



(b) educational R&D centers and laboratories; (c) schools of education 
which would use the modules as instructional materials; and (d) non- 
educational R&D agencies. Depending on the scope and intensity of the 
dissemination effort, a potential demand for 100 to 500 copies of each 
of the individual modules would be a reasonable estimate during 
the first two years of dissemination. One library copy of a module can, 
of course, be "used" by multiple borrowers. 

Guides . Support materials, including guides to the (a) Instructional 
System, (b) Quality Control of the Instructi onal Systeiii , (c) Pre-servi ce 
Program implementation and (d) Continuing Education Program implementation, 
and catalogs of instructional resources and of competence assessment 
instruments are being developed to support the comprehensive use of the 
Functional Competence Training Program (item (a) above). A detailed 
description of each of these items will be found on pages 60 to 68 of this 
report. 

C ompetence assessment batteries . Another facilitative device to 
encourage broad utilization is the Competence Assessment System, which 
provides individuals, R&D agencies, and training institutions with a 
method of assessing competence in the technical areas of educational DD&E. 
The system delineates DD&E competencies by level of proficiency, provides 
methods for assessing competence in trainees, and suggests criteria for 
certifying competence. A validated system of competence assessment has a 
utility far beyond its use with the Instructional System. It can become 
an instrument that employers can use to screen applicants, to place new 
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employees; to assign employees to tasks, to identify employee competence 
deficiencies, and to plan a personnel development program. It is 
anticipated that, during the first two years of dissemination, 15 to 30 
agencies vn'll make some use of the competence assessment batteries. 
3. Uniqueness of Programs and Materials 

In addition to the design cha.-uCter'Stics and practices reported in the 
f'lrst part of this section, there are four unique aspects of our program. 

Functional competence base . The design of instruction and the 
objectives specified in each of the learning episodes have a skilN 
related focus. This orientation is, perhaps, the most notev/orthy feature 
of the DD&E training. Role-relevant competencies are identified prior 
to the development of instructional resources. Careful empirical analysis 
of the DD&E task list, obtained in preliminary studies, serves as the 
foundation on which performance objectives and learning episodes are 
built. Purposeful activities, sequenced in time and organized into a 
situational scheme, can be planned so as to become the learner's own 
program. The individual will operate his/her own learning system, will 
always know what activities he/she is now and will become involved 
in and why, will perceive how far he/she has advanced at any given time, 
and will understand what has yet to be achieved to give evidence of 
additional progress. 

The Engineered Internship . The concept of the Engineered Internship 
will place the learner, his Instructional Resource Manager, and his work 
supervisor in a new and unique relationship, far removed from the ^^^^^ditional 
student role. The functional context of a job combined with flexibly . 



scheduled, self-administered, job ;-elevant instruction puts the highest 
possible emphasis on the trainee's acquisition of competencies that should 
be useful in his/her job and that are important to the trainee. With the 
help of two experienced and interested people, the learner can select and 
then master those competence elements, sets, and subsets which will be of 
greatest benefit to him/her. The work he/she does is not forced into the 
rigid semester/quarter mold of an on-campus course. The trainee progresses 
at his/her own speed, may work full or part time, and may apply previously 
learned skills to gain academic credit. All this will occur within the 
general context of applying each competence element to real problems, 
tasks, or projects. The Engineered Internship will help bridge the gap 
between academic and on-the-job learning, incorporating many factors 
recommended in exemplary .training designs. Its conceptual strength and 
consistency appeal to If.arner, teacher, and work supervisor. 

Materials development. The development of instructional resources 
for the achievement of competence in DD&E follows a materials development 
cycle through which the Far West Laboratory has carried mony products to 
national attention and use. Careful formulation is made of specifications 
and criteria for judging the adequacy of the learner's response. Prototype, 
field, and operational forms are developed and tryouts conducted, 
accompanied by continuous assessment activities. The materials produced 
by the Consortium will be developed to a point where they will be ready 
for extensive operational testing. 

Making use of other learning re5;n.irrp<;_ The many projects funded over 
the past few years by the Research Training Branch (now NIE/TFRT) offer 
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additional DD&E-focused learning resources. If they are within the scope 
of our competence sets, all such materials, regardless of source, may be 
incorporated into our training system. Efforts will be made to avoid 
duplication of effort and redundancy. 

4. Efforts to Develop Cooperation among R&D Training Projects 

One significant outcome, going beyond this Consortium project, has 
been the interaction among project staffs in the three model consortia*''\-riJ 
other research training projects. During the year, visits and exchange of 
information and materials have intensified interaction among the three 
Consortia. Glen Heathers of the Pittsburgh Consortium has visited our 
program and discussed with us mutual use of materials. Dan Stufflebeam 
of the Ohio State Consortium visited twice and has been involved in the 
critique of our Evaluation Series. Of other research training projects, we 
have had direct exchange with Michael Scriven of the University of California, 
Berkeley, and Skip McCann of the UCLA training project. The conference of 
Consortia and research training projv^ct staffs co-sponsored by the three Con- 
sortia, held in December, 1972, at the Ohio State University, gave impetus to 
this growing cooperation. The R&D training information system under design at 
Far West Laboratory may facilitate continued coordination and exchange 
of information and materials. 



II. 1972 ACCOMPLISHMENTS 

The goal of the Far West Consortium is to develop and validate a 
transportable, competence-based system for the training of entry-level 
professionals in educational development, dissemination and evaluation 
(DD&E). To reach this goal, five tasks must be perfoniied: 

1- Systems Development. Development of the Instructional 
. System, the Competence Assessment System, and the Quality 
Control stystem. 

2. Product Development. Development and validation of the 
materials and instruments that provide the resources for 
the Instructional and CoT'petence Assessment Systems. 

3. Training Implementatinn Creation of suitable test 
conditions in which training cm be implemented and 
evaluated. 

^- Pissemination/Utilizatio n Planning . Planning and arranging 
for the dissemination and utilization of the products and 
systems that have been developed. 

5. Project Management. Management of the four tasks described 
above. 

1972 accomplishments are reported within the structure of the 
tasks outlined above. 
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TASK ONE: SYSTEMS DEVELOPKiiHT 

Systems development involves the conceptualization, design, 
formative development, field testing, evaluation, and revision of the 
Functional Competence Training System. The operational model of the 
training system is displayed in Figure 1 on the next page-. The central 
scheme of the model is presented in the upper right section of the figure. 
The scheme projects the interface and interaction of the learner system, 
the instructional.resources and the application systems. The individual 
enters the training system by planning his/her program with the Instruc- 
tional Resource Manager and the work supervisor. Competence is acquired as 
the trainee, guided by the Instructional Resource Manager, completes 
selected modules and. he/she then applies these competences to the job, 
under the guidance. of the work supervisor. 

The instructional resource- a.e developed to satisfy the 
requirements specified for the competence sets, subsets and elements 
constituting the basis of the MtructionalJ^. The same competence sets, 
subsets, and elements also provrde the basis of development of the compe- 
tence assessment batteries, sub-batteries and items that constitute 
Competence A ssessment Sy <;tPrTi 

IheXuiLctionaUompetence Trai^ ^lston^_Entr^ p,ofessional Level 
The Functional Competence Training System is designed to provide ~ 
users with instructional resources and procedures needed to acquire 
technical skills in the following competence areas: Planning and Design; 
Infomation/Data Collection; Comn,uni cation Skills; Developmental Engineering; 
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Evaluation; Analysis and Definition; Dissemination and Marketing; 

and Management. (A description of the competence area is provided 

in Appendix C). The minimal level of competence has been set equal 

to that of a DD&E junior professional with an MA degree and one year of 

DD&E experience or of someone with a BA degree and two to three years DD&E 

experience. The training system provides opportunities for application 

of acquired competence in the real contexts of DD&D through the 

engineered Internship and for assessment and certification of the attained 

competence. 

Developing a flexible approach . While the design and training has, of 
necessity, been influenced by institutional constraints, emphasis has 
been on attempting to change academic institutions to test the 
training and designed. This is seen in attempts to move away from 
structured "courses" and allow students flexibility in selecting 
instructional m;^terial$ coverinn competenc^^ related to thpir .^ob 
requirements. Negotiations with California State University at San 
Francisco (CSUSF) have resulted in decisions permitting students to 
select modjles from various functional context series. Modules 
have been assigned credit hours so that all that is required is 
that sufficient modules will be selected to satisfy the institution's 
course requirements (three hours per course). Arrangements have also been 
made which will permit students to receive one credit hour per application 
of competencies on the job for each course. Although these arrangements 
may seem trivial, they represent a major step for the academic institution. 

Engineered Internship . The Engineered Internship program is the key 
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operational element of the training system. Prototype testing of the 
Engineered Internship program was initiated at CSUSF in Fall 1972 with 
17 newly enrolled students and 14 continuing students. The heart of 
this program is the functioning of the triad of trainee. Instructional 
Resource Manager (IRM), and the work supervisor. The IRM guides the 
operation of the triad, A more detailed description of the Engineered 
Internship is found on page 14 of the Analysis of Projected Impact. 

Instructional Resource Manager's Guide . During Summer 1972, the 
initial form of the Instructional Resource Manager's Guide was developed. 
This guide is a comprehensive manual designed to instruct the Instructional 
Resource Manager (IRM) in his role of implementing the DD&E training system. 
The guide explains this unique training system, and specifies responsi- 
bilities the IRiM has to the trainee and v/ork supervisor. 

The Guide has four parts. Part I describes the role of the 
IRT^, explains the DD&E training system* and details the IRM's responsibi- 
lities. Part II describes procedures for assessing student competence. 
Included is a Student Competency Matrix which graphically connects required 
competence with appropriate modules. Examples are given to illustrate 
use of this matrix. Part III is the Internship Monitoring System. It explains 
how to determine trainee needs and how to prepare "work plans" with trainees and 
work supervisors. Sample forms for these evaluation and monitoring procedures 
are included. Part IV is a Catalog of Instructional Mater als in three 
sections. Section A lists the instructional materials availab'le to the 
IRM and trainee. These materials include not only the modules of program 
materials, but also related and accessory materials. Section B is an over- 
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view of each module. Section C is a detailed description of each module. 

Other Guides . Based upon initial use of the Instructional Resource 
Manager*s Guide during Fall 1972, the decision was made to replace it 
with a set of guides and catalogs for a variety of uses. As planned these 
will include the Guide to the Instructional System , Catalog of Instructional 
Resources , Guide to the Implementation of the Pre-service P r ogram , Guide 
t o the Implementation of the Continuing Education Program , Guide to the 
Competence Assessment System , and the Catalog of Competence Assessment 
Instruments . These guides and catalogs are described in the 1973 Work Plan. 

Evaluation . During the year we have evolved a functionally differen- 
tiated scheme of evaluation. Accordingly, evaluation concern subsumes 
(1) the formative evaluation of instructional materials, (2) the formative 
evaluation of the Engineered Internship, (3) the assessment of student 
competence, and (4) the quality control of both the Instructional and 
the Competence Assessment Systems. 

Formative evaluation of instructional materials . Analysis of the 
prototype modules and examination of the procedures used to evaluate 
their adequacy have indicated a need to establish a more thorough method 
of providing data on the students' use of materials and other feedback 
to be used in revision. Accordingly, durirg 1972, a restructured 
formative evaluation system has been instituted which includes (1) the 
involvement of at least two subjects (representative of the student population) 
during the development of the prototype form of modules, (2) student 
feedback on quality and effectiveness of the modules. (Such feedback is 
acquired during prototype testing of the module involving at least five 
students and field testing with 10 to 20 users.) (3) Instructional 
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Resource Managers' evaluation of modules, (4) outside expert and (5) staff 
review of modules. Forms and guidelines have been developed to implement 

the procedures described here. A more detailed description of this 
procedure is reported, on page 28« 

Formative evaluation of Engineered Internship . A monitoring system 
has been implemented during Fall 1972 to provide formative evaluative 
information on the Engineered Internship. This system provides feedback 
on r.n3 functioning of the triad (trainee, Instructional Resource Manager, 
work supervisor), on the development of work plans for trainees, and on 
the degree to which modular instruction correlates with on-the-job 
experience. 

Assessment of student competence . A major development late in 1972 
was .a move toward the construction of competence assessment instruments 
which will be used as both pre-and post-training means of assessing 
student competence in the competence areas. These instruments will 
also provide for the certification of competence without formal training. 
Organized into sets of batteries, the instruments will be applicable to 
specific competence areas and subsets of the instruments to particular modules. 

First drafts of initial subsets of competence assessment batteries 
(module level) have been developed and a plan for validating and 
calibrating the instruments has also been prepared. For more details, see 
Attachment One. 

Quality control . Quality control provides means and methods by which 
implementing agencies may monitor and evaluate the adequacy of use of ■ 
the Instructional and Competence Assessment Systems. The quality control 
concepts we have developed call for construction of guides which specify 
(1) the questions that neeo to be asked about these systems, (2) the 
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information and data that are to be collected, (3) the instruments 
with which to collect the information and data, and (4) the viay to 
interpret the information" and data one ? collected. 

During the year, the evaluation activities of the project have been 
reviewed by Dr. George Temp, Dr. Daniel Stufflebeam, Dr. Maurice Eash, 
and by the site review team. (Comments of the consultants are reported 
in Appendix G.) These reviews have provided valuable assistance in the 
development of the new approaches to evaluation. 



TASK TWO: PRODUCT DEVELOPMENT 

Product development involves the specification, selection for design, 
formative development, field testing, evaluation, and revision of the 
materials and instruments constituting the required resources of the 
Instructional System and the Competence Assessment System. Accomplish- 
ments wili be reported on the development of instructional resources and 
the development of competence assessment instruments. 

Development of Instructional Resources 
Instructional resources are now being developed at the entry professi 
level for six of the eight competence areas. The instructional materials 
for each competence area are called a series. The series are: (1) 
Planning and Design, (2) Information/Data Collection, (3) Communication 
Skills, (4) Developmental Engineering, (5) Evaluation, (6) Dissemination 
and Marketing. Planning for developmental work was started on two 
competence areas. Analysis and Definition, and Management. Series are 
made up of instructional modules, each of which focuses on a cluster of 
competence elements. Modules are referred to by numbers within their 
series — Module 3.2 is the second module of Series 3, Communication 
Skills. Witbin modules, learning activities are grouped by episodes which 
attend to specific competence or skills within the set. Each episode 
has its own objectives, learning activities and progress checkpoints that 
allow the trainee. Instructional Resource Manager, and work supervisor to 
evaluate the trainee's progress. Typically, modules are designed to be 
self-instructional, but one trainee may also work with another or in a 
small group. 
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Modules are designed around specific performance objectives derived 
from both logical and empirical task analyses. They are designed to enable 
the learner to achieve a pre-determined level of competence. A definition 
of levels of competence appears in Appendix G . Each module is designed 
in relation to others in a series. In several modules, relationships across 
series have been incorporated, eg. "Planning for Evaluation" (Module 1.4 
in the Planning and Design Series) is also an integral part of the Evalua- 
tion Series. 

The specific developmental tasks are grouped into seven phases: 
Phase 1. Prepare developmental plan for series, 

assign competence, outl ine content. 
Phase 2. Prepare prototype form. 
Phase 3. Test prototype form. 
Phase 4. Prepare field test form. 
Phase 5. Test field test form. 
Phase 6. Prepare operational form. 
Phase 7. Disseminate operational form. 
Phase 1 covers the design and organizational effort. It includes the • 
allocation of competence identified in the initial Design Report and 
the Competence Matrix analysis and an outline of content developed by 
subject matter specialists, author(s), and product development staff. 
In this phase, authors receive a Module Developer's Kit which provides 
specific instructions and performance criteria for each facet of development. 

Phase 2 of the production effort places the prototype form of the 
module in the hands of students and reviev/ers. During development of the 
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prototype form, as parts of a module are developed, they are tried out 

with two or three trainees or subjects representative of the trainee population. 

Phase 3 provides for prototype testing of the module with the 
involvement of five or more traihees, and includes such additional 
evaluative activities as reviews by content specialists, developmental 
staff and work supervisors. 

Phases 4 and 5 repeat the revision and testing cycles. The 
revised module is given to 10 to 20 more users. Their evaluations form 
the basis for the final revision. Phase 6 , and the printing and dissemina- 
tion of the operational form. Phase 7 , Note that operational testing 
in the sense in which that term is used by the Far West Laboratory is not 
providedjT 



During 1972, production control procedures have been revised and 
applied to provide better information on the status of modules under 
development. Tools have also been developed to implement procedures, 
such as the Module Review Checklist and Module Developer's Kit and 
styleguide. 

The production control procedures include several specific check- 
points involving the analysis of student use data^ consideration of the 
observations of the Instructional Resource Managers and work supervisors, 
and staff and expert review, 



In the Far West Laboratory 's development cycle, operational testing involves 
extensive field use without direct developer involvement . Time and resources 
available preclude this last step. All products will be developed up to a 
point where the operational form is ready for testing. An additional one to 
two years would be required to accomplish operational testing of all products. 
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Approaches developed to aid in monitoring developmental progress include 
the following: 

a. A ^^dule Developer's Kit , which accompanies author contracts 
or staff assignments, contains suggestions for establishing 
objectives, organizing content, preparing active-response 
exercises and progress checkpoints, and giving interpretations 
of student responses in feedback sections. 

b. A 31 -item Module Review Checklist that enables both 
Laboratory and outside expert reviewers to analyze each 
instructional product has been piloted, tested, and revised; 
it is now used routinely. This checklist standardizes 
examination, comment, and recommendations for each product. 
It gives authors further insight into specifications, 
evaluation techniques, and overall design requirements. 

c. Guidel ines for securing orderly, consistent application of 
the formats, structure, and nomenclature of the Consortium's 
products and a manual of style have been adopted to assure 
high quality in the copyediting of all publications. 

d. A materials control specialist was employed in Fall 1972 
to give direction to all phases of production, printing, 

and distribution, including copyright questions and graphics. 
This specialized function will greatly add to the accepta- 
bility and attractiveness of all products in the future. 

e. Annual , quarterly, and monthly milestone charts , accompanied 
by progress reports to program and Laboratory management. 



help to prevent delays, to review progress frequently, and 

to pinpoint difficulties in early stages. 
Status of Product Development 

Accomplishments in product development are reported here relevant 
to the entry professional and paraprofessional programs. A comparative 
analysis of planned and actual accomplishments for 1972 are reported in 
Appendix K. 

Entry Professional Program . Accomplishments at the entry 
professional level are shown in Figure 2. The figure provides a 
comparison of accomplishments by the end of 1971 and the end of 1972. It 
also gives a projection of status by May 31, 1973. A summary comment, 
by series, follows. 

Series 1: PLANNING AND DESIGN. User information and staff review 
have indicated general adequacy of content. Series is now under revision 
for field test form. 

Series 2: IN FORMAT I ON /DATA COLLECTION AND ORGANIZATION. User 
information and staff review have indicated general adequacy of the 
content of the modules in this series except 2.4, which will be 
redesigned. Module 2.1 is in field testing, the remainder of the series 
is now under revision for field test form. 

Series 3: COMMUNICATION SKILLS. This entire series has been 
redesigned. The former Module 3.1, "Communication in a DD&E Agency", has 
been removed and will serve as a reference. Module 3.2, "Listening and 
Speaking," is now Module 3.1; Module 3.3, "Consumer-Oriented Information," 
has been subsumed under the Dissemination and Marketing Series; and Module 
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3.4, "Technical Writing," has been divided into three separate modules. 
The entire series is now under revision for field test form. 

Series 4: DEVELOPMENTAL ENGINEERING. Review of prototype forms of 
modules in this series has indicated general adequacy of the content and 
methods of presentation. Module 4.3 will be redesigned to better meet 
specifications. Three modules are now in prototype testing; two are nearing 
completion of their prototype forms. 

Series 5: EVALUATION. This series has been redesigned following 
prototype testing of Modules 5.1, 5.2, and parts of 5.3 and 5.4, and 
expert and staff review of the entire series. Two modules are in 
revision for field testing, three modules are in prototype development. 

Series 6: ANALYSIS AND DEFINITION. This series is in the initial stage 
of planning for development. 

Series 7: DISSEMINATION AI^D MARKETING. This series was redesigned 
to achieve a better correspondence with competence scheme and design 
specifications. One module is now in prototype testing, and two modules 
are in the initial stage of prototype development. 

Series 8: MANAGEMENT. This series is in the initial stage of 
planning for development. 

P^ra^rofesslo^^ T„e following materials are available 

at the paraprofessional level: 

Series 1: PLANNING AND DESIGN. Prototype forms of Modules 1.1. 
1.2, 1.3, and 1.4 were developed and printed; prototypes of Modules 1.5 
and 1.6 are in draft form. 

Series 2: INFORMATION/DATA COLLECTION AND ORGANIZATION. Prototype 
forms of six modules were developed and printed. 



Series 3: COMMUNICATION SKILLS, Prototype forms of four modules 
were developed and printed. 

Series 4: DEVELOPMENTAL ENGINEERING. First drafts of prototype 
forms of five modules are developed. 

Series 5: EVALUATION, First drafts of prototype forms of six 
modules are developed. 

Developmental activities with the paraprofessional program were 
halted in Octvober 1972 in compliance with the recommendations of the NIE 
site- review team. 
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B. Development of Competence Assessment Instruments 

The prototype evaluation system prepared by Educational Testing 
Service, dated September 1 , 1971, and included in the Final Report for 
the period 2/1/71 to 12/18/71 was subjected to critical analyses by an 
external evaluation panel consisting of Sam Sieber of Columbia University, * 
David Clark of Indiana University, and Derek Nunney of Oakland Community 
College, Michigan. In addition to these reviews, a comprehensive 
analysis was made of the Consortium's evaluation requirements by George 
Temp and by Maurice Eash and Herbert Walberg of the Office of Evaluation 
Research of the University of Illinois (see Appendix F). A follow-up 
visit was made by Maurice Eash on July 11, 1972. Based upon the 
comments and recommendations in the above-mentioned reviews, a new 
approach to assessing trainee competence was planned. This planning 
required a reexamination of the competence lists whic"! were presented 
in the original Design Report. This reassessment was for the purpose of 
organizing the various competences into a matrix for use in determining 
student entry level and needs, developing wor!^:splans for internships, and 
assessing trainee progress. (See Figure 3) 

During Fall 1972 a comprehensive approach for assessing student 
conpetence was designed. Exemplary competence assessment techniques for 
Modules 3.2 and 4.2 were developed in initial form. These assessment 
techniques will constitute the competence assessment battery for each 
series of modules and consist of: (1) Trainee Self-Rating Scale, (2) Super- 
visor Rating Scale, (3) Job Knowledge Test, (4) Simulation Exercise, and (5) 
Product Rating Scale. These techniques are described on page 76. 
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Standard procedures for test development and validation will be 
followed as far as practical, given the relatively small sizes of the 
trainee groups and the target level calibration groups (20 to 40 persons 
in each group). 

In addition to the two first drafts of exemplary competence assessment 
batteries for Modules 2.3, and 4.2, first drafts of batteries have been 
prepared for Modules 1.6 and 2.1 and first drafts of batteries for Modules 
1.1, 1.2, 4.1, 4.3, 4.4 and 7.1 are under development.. 

In Attachment One we present a description of the assessment 
sequence and a sample of a module-related set of competence assessment 
instruments. 
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TASK THREE: TRAINING IMPLEMENTATION 

During 1972, DD&E training was continued at four training 
institutions, at entry professional level at California State 
University at San Francisco (CSUSF) and paraprofessional level at Canada, 
Merritt, and Contra Costa Colleges. 

The entry professional training program in education with a concentra- 
tion in DD&E, leading to the MA degree, is currently in operation at 
CSUSF. A Program Coordinator handles the administrative aspects of program 
implementation and operation, and arranges internship positions for 
trainees. Thirty-one trainees are currently enrolled'in this program — 
14 Cycle 1 trainees (entered in Fall 1971) and 17 Cycle II trainees (entered 
in Fall 1972). All trainees have been placed in internships, with 
school districts (14), R&D agencies (15), or industrial training establish- 
ments (2). All trainees have been assigned to one of three Instructional 
Resource Managers who assist them and their work supervisors in formulating 
work plans, arranqinq study plans, monitoring progress, and providing feedback 
on trainee work. Trainees may select modules now developed and/or other relevant 
training materials to satisfy their competence training requirements. 

The three Instructional Resource Managers were assigned to provide 
instructional services as follows: 

Spring Semester: Developmental Engineering - Dr. Earl Miller 
Communication Skills - Dr. John DeCecco 
Evaluation - Dr. Harold Jonsson 
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Fall Semester: Planning and Design - Dr. Norm Wallen 

Information/Data Collection - Dr. Enoch Sawin 
Engineered Internships - Dr. Wall en, Dr. Sawin, and Dr. Hal 
Vitae on these instructional personnel is presented in Appendix D. 

The paraprofessional training was implemented through offering DD&E 
courses in five competence areas: (1) Planning and Design; (2) Information/ 
Data Collection, (3) Communication Skills, (4) Developmental Engineering 
and (5) Evaluation. 

Approval was obtained for Consortium courses at Canada College during 
Spring and Fall semesters, 1972. Instructional Resource Managers were 
selected and assigned to provide instructional services as follows: 
Spring Semester: Developmental Engineering - Mr. Kennedy 

Evalua\:ion - Mr. James Upton 
Fall Semester: Planning and Design - Mr. Ken Kennedy 

Information/Data Collection - Mr. Jumes Upton 
Communication Skills - Mr. Kilpack 
Vitae on these instructors are presented in Appendix D. 

Employees of Far West Laboratory have been enrolled in the DD&E 
program through Merritt College. Instructional experiences are managed 
by accredited Laboratory staff and instructional facilities are provided 
in the Learning Laboratory. These students are studying materials in the 
Communication Skills, I nfomaati on/Data, and Planning and Design series. 

Eight Concentrated, Employment Program trainees employed by Far West 
Laboratory have been enrolled in DD&E programs through Contra Costa College. 
Again, instructional materials, facilities, and services are available at 
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Far West Laboratory for these students* Students in this program have 
been enrolled in courses representing the five competence areas mentioned 
above. A summary account of students enrolled in the various programs is 
provided in Appendix E • 

There are four additional aspects of training implementation reported 
here; (1) student personnel services, (2) staff orientation and coordina- 
tion, (3) employee recruitment and (^) supervisor orientation* 

Student personnel services * Student personnel services include the 
recruitment and orientation of students and guidance and counseling 
services* During the year, 21 students v/ere recruited and enrolled in the 
DD&E program at CSUSF, three at Canada College, five at Merritt College, and 
eight at Contra Costa College* Students in the entry professional program 
at CSUSF are assigned an advisor who handles counseling services* Students 
in the program have made no requests for changes in advisors or indicated 
any dissatisfaction with the services provided* Students in the para- 
professional program at the various community colleges are assigned regular 
members of the counseling staff* 

Staff orientation and coordination * During the year occasional 
coordination meetings have been held for informal sharing of experiences 
with instructional methods and problem areas* These meetings provided 
information which supplemented the formal evaluation data derived through 
regular evaluation procedures* 

A series of orientation meetings were held at CSUSF and Canada College 
before the start of the Fall semester* These meetings resulted in more 
clearly defining the role of the Instructional Resource Manager as contrasted 
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with a classroom instructor, and introduced procedures and tools to be 
used in implementing the Engineered Internship program. 

Employer recruitment . During the late spring and summer months, 
recruitment of employers was undertaken to secure internship positions in 
the fall. Contracts were negotiated with the program coordinators (Dr. 
Geroge Hallowitz, CSUSF and Dr. Robert Bennett, Canada College) to contact 
employment agencies and locate paid or unpaid internships for students at 
their respective institutions. Fourteen internships have been located for 
the entry professional program and three for the paraprofessional program. 

Supervisor orientation . The program coordinators conducted orientation 
of supervisors at agencies providing internships. Supervisors were briefed 
on the Functional Competence Training System, with specific information 
on the Engineered Internship. Instructional Resource Managers held a 
series of meetings with agency supervisors to discuss the Engineered 
Internship program. The competencies which students were acquiring were 
reviewed and clarified. In many instances, specific work tasks were 
suggested to provide on-the-job application of various competencies. The 
operation of the triad --trainee, Instructional Resource Manager, and work 
supervisor --was discussed and clarified. Records of these meetings were 
kept, indicating dates, who attended, and general topics discussed. Also, 
supervisors at agencies providing internships were given the forms necessary 
to provide monitoring and evaluation information on the Engineered Intern- 
ships. Instruction was given as to the proper use of these forms and sub- 
mission of information. 
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Learning Laboratory. During the year two learning laboratories 
were maintained for use by students and Instructional Resource Managers; 
one on campus at. Canada College, and one at the Far West Laboratory. 

Each learning laboratory serves as a meeting place for students and 
IRMs, as a place for study, and as a resource for materials, special ERIC 
collections, etc. In addition, each laboratory provides study carrels, 
drafting equipment, calculators, microfiche readers, and other equipment 
for use by students and IRMs. Special laboratory hours were set up 
to accommodate users in the late afternoon and on Saturdays. Laboratories 
were manned by trained personnel who provided assistance to users as 
needed. In addition, the Far West Laboratory library and the Educational 
RDD&E Personnel and Training Library, located adjacent to the learning 
laboratory, provide additional relevant materials for users of the 
learning laboratory. 
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TASK FOUR: DISSEMINATION/UTILIZATION 

Dissemination/Utilization involves preparation of materials 
developed by the Consortium into an operational, transportable form, 
and arrangements for the use of those materials by training institutions, 
R&D agencies, and others both inside and outside the geographical region 
of the Consortium. 

In the Analysis of Projected Impact, page 11, we projected three 
levels of use: the comprehensive training system; 'a variety of configurati 
of parts of the system; and use of the smallest independent elements, the 
training modules and relevant assessment instruments. 

Another dimension of use is related to the mode of use.. Two major 
implementation modes have already been planned: pre-service and continuing 
education. 

The Pre-service Progra m is degree-oriented and set in a college or 
university with a graduate program willing to sponsor educational DD&E 
as an area of specialization. Students accepted into the program will 
already hold Bachelor's degrees and will have the MA as their goal. Once 
accepted for the program, trainees will be placed in internships at DD&E 
agencies or other suitable work settings. Each trainee will be assigned 
to an Instructional Resource Manager who will assist the trainee and 
his/her work supervisor in planning internship activities and assigning 
pertinent materials from the Instructional System. (It is also quite 
possible that a program could be implemented which would grant BA degrees 
in Educational Development. In Cal ifornia, colleges of education 
award BA degrees, hence, this alternative has been untestable). 
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Variations of this model are extension courses and "mater ials-or- 
resource-based" programs in a variety of educational professional develop- 
ment .programs, conducted at institutions of higher education, in cooperation 
with R&.O agencies. 

The Continuing Education Program can be implemented by one or more 
R&D agencies with employees who require training in competence covered 
by the Functional Competence Training System. The number of modules used 
and whether or not credit is awarded will depend on the needs of the 
individual student and the availability of an accrediting institution. 

Other modes of use may include the development and implementation of 
workshop, institute, or learning-team forms and self-directed independent 
use of niaterials. 

Some preliminary plans for dissemination and utilization have been 
developed by the Long Range Planning Committee of the Consortium -and are 
reported in Appendix I. 
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TASK FIVE: MANAGEMENT 

Management involves the coordination of the affairs of the 
Consortium, the management of the in-house staff assigned to the DD&E 
project and cos>t accounting. 

Consortium activities. Agencies of the Consortium have been 
identified in Appendix A. During the year member R&D agencies have 
cooperated in the development of training programs. Member training 
institutions have aided in implementing the various programs. The 
Consortium Board was convened on March 14, 1972, to review Consortium 
operations, discuss priorities and revisions to the schedule of accomplish- 
ments, and to review reports from external review consultants. Activities 
scheduled for 1972 were reviewed at this meeting. The decision to delay 
development of instructional materials in the Analysis and the Management 
competence areas was made to provide the staff and resources necessary for 
improvement of evaluation procedures, instructional resources management 
practices, and Engineered Internship procedures. A series of coordination 
meetings was held with various Consortium members to negotiate contracts 
for accomplishing scheduled development activities. Two meetings of the 
Consortium Planning Committee were convened to develop plans for continua- 
tion of the Consortium beyond August 31, 1974 and for dissemination of the 
Consortium's products. These meetings resulted in the set of preliminary 
plans outlined in Appendix I. 

The use cf subcontractors and consultants has always been an essential 
part of Consortium planning. During program year 1972, subcontracts have 
been awarded to AIR, to HumRRO, to Hal Jonsson, Enoch Sawin, and John DeCecco 
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of CSUSF and to Ken Kennedy and James Upton of CarTada College, and 
William Wolf, University of Massachusetts, for development of training 
materials, and to George Hallowitz of CSUSF and Robert Bennett of Car^ada 
College for development of implementation systems. Maurice Eash, Herbert 
Walberg, George Temp, and Daniel Stufflebeam have advised us in the area 
of evaluation. Eva Baker provided consultation on materials development 
procedures. 

During the first part of the year we made an analysis of the role 
subcontractors and consultants should play. We concluded that: 
(1) analysis and design' functions can probably best be accomplished by 
inhouse staff, with regular review by outside consultants; (2) materials 
development can be either subcontracted or carried out by inhouse staff, 
depending on available skills and assessment of cost effectiveness; 
(3) materials should be reviewed by both outside experts and the central 
staff; (4) revision of materials can be subcontracted or done by staff, 
depending on skills and costs; (5) uniform technical and stylistic editing 
should be done by staff; and (6) though we should maintain an inhouse 
evaluation capability, we should also use- external evaluation. Systems 
development can be done by staff, with occasional involvement of 
consultants for review purposes. 

Staff management . At the beginning of the year the inhouse staff 
consisted of the Principal Investigator (part-time), the Project Director 
(Joe Ward), the Senior Evaluator (Freeman Elzey), three half-time interns, 
the project secretary, and a part time secretary. A senior developer 
(John Hourigan) was added to the staff in February. 

During the first part of this program year, an analysis v/as conducted 
to find out if all functions necessary to attain program objectives were 
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being fulfilled and if personnel were sufficient in numbers, organized 
appropriately, and using adequate methods to carry out their functions. 
Our analysis indicated that the formative development of materials and 
competence assessment instruments and the development of the Engineered 
Internship had not yet been adequately accomplished because of the 
complexity of launching these program efforts and the inadequacy of 
resources. It appeared that most of these functions should be carried out 
by central staff, since close and frequent conmunication is necessary to 
assure their successful completion. 

By mid-June, with the addition of several staff members, the 
Instructional and Training Systems program was organized, with its primary 
responsibility the DD&E project. The staff was organized into four functional 
components: (a) analysis and design, (b) development, (c) evaluation, 
and (d) field services and dissemination. A senior staff member assumed 
coordination of each of the components -Banathy, Rosenoff, Elzey. and Ward 
respectively. Project management has been carried out by a management 
team consisting of the Program Director (Bela Banathy) and the three other 
component heads. The management team has met weekly to plan, monitor, 
and control the projects. Minutes of these meetings, and monthly progress 
reports have been submitted to the Principal Investigator (Paul Hood) who 
is in daily contact with the Program Director. In addition, the Principal 
Investigator has met regularly with senior staff members. 

Cost accounting system. A system of cost codes provides detailed 
data on the actual costs of various activities undertaken. Budget and 
accounting procedures are administered by the Far West Laboratory's 
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Business Office. Budget projectiuns are now made on a monthly basis. 
Printouts of current accumulated expenditures are provided monthly and 
are compared with our monthly projections. In late 1972, s new cost code 
scheme was developed in line with the program revision and reflects the 
1973 work plans. 



Section Two : WORK PLANS FOR JANUARY 1 - MAY 31 , 1973 



ERIC 



INTRODUCTION 

This Work Plan for the further development of the Functional Competence 
Training Program covers the period from January 1 to May 31, 1973. Plans 
are reported in terms of • the five major tasks that must be performed: 
Systems Development . Development of the Instructional System, 
the Competence Assessment System, and the Quality Control System. 

2- Product Development . Development and validation of the materials 
and instruments that provide the resources for the Instructional 
and Competence Assessment Systems. 

3- Training Implementation . Creation of suitable test conditions 
in which training can be implemented and evaluated. 

4- Dissemination/Utilization Planning . Planning anc" arranging for 
the dissemination and utilization of the products and systems 
that have been developed. 

5. Project Management . Management of the four tasks described 
above. 

For each of these five tasks, this Work Plan projects specific 
accomplishments for three milestone dates: April 1, 1973, May 31, 1973, and 
August 31 , 1974. This organizatior. provides time frames in compliance 
with the requirements of the Researcher Training Task Force of NIE. A 
statement of definition and a description of anticipated accomplishments 
are presented for each of the five task areas. 
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TASK ONE: SYSTEMS DEVELOPMENT 

Systems development involves the conceptualization, design and 
formative development, field testing, evaluation and revision of the 
Functional Competence Training System and its three component systems: 
(a) the Instructional System, (b) the Competence Assessment System, and 
(c) the Quality Control System. 

The Functional Competence Training System 
The Functional Competence Training System is designed to provide 
users with instructional resources and procedures needed to acquire 
entry-professional level technical skills in the following competence 
areas: Planning and Design; Information/Data Collection; Communication 
Skills; Developmental Engineering; Evaluation; Analysis and Design; 
Dissemination and Marketing; Management; and the Educational 
Information Consultant.© (See Appendix C for description of ccmpetence 
areas.) The level of competence attainable will equal that of a DD&E 
junior professional with an MA degree and one year of DD&E experience or 
someone with a BA degree and two to three years' DD&E experience. The 
training system provides opportunities for the application of the acquired 
competences in the "real" contexts of DD&E through the Engineered Intern- 
ship and for assessment and certification of the attained competences. 
Products relevant to the Functional Competence Training System are the 
Design of a Functional Competence Training System and the Final Report 
on the Development of the Functional Competence Training System . 

^his series consists of three modules adopted from a recently completed 
instructional system called the Educational Information Consultant. These 
modules are compatible with both the Information/Data or Communication Skills 
Series. For accountability reasons, however, vie keep them separate and have 
designated them the ninth serie? in the system. Appendix L gives a detailed 
description of this series. 
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Anticipated Accomplishments : 

1. T he Design of a Functional Competence Training System . This 
document will describe the design, organizing concepts, and essential 
characteristics of the program (a major revision of the original design). 
By May 31, 1973, an outline will be developed and by August 31, 1974* 
the document will be available for dissemination. 

2 . Final Report on the Development of the Functional Competence 
Training System . This report will document the Consortium's experience 
in development, val idation, and dissemination of the Functional Competence 
Training System so the knowledge and procedures may be communicated to 
others concerned with training development in general and educational 

R&D training in particular. A printed report will be submitted by 
August 31, 1974, 
A. The Instructional System 

The Instructional System provides users with validated instructional 
resources and procedures. Resources will be in the form of a series of 
self-administered instructional modules developed for each DD&E competence 
area described in detail in Task Two: Product Development . Procedures 
are designed and validated by which individual learners can acquire DD&E 
competences using the instructional resources and apply these 
competences in a "real" educational DD&E context through an Engineered 
Internship. Special emphasis will be placed on studying the variables 
involved in the operation of the Engineered Internship and examination of 
interaction patterns among the trainee, the Instructional Resource Manager 
and the work supervisor. Alternative configurations of use of the Engineered 
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Internship will also be explored (see Appendix J). Two implementatit 
models of the Instructional System will be developed: the pre-service 
implementation model (degree-oriented) and the continuing education 
implementation model (in-service). Products of the Instructional System 
will include the instructional re<;o„rrp<:r a Guide to the Instructional 
^i^^tem; a Ca talog of Instructional ResQurrP<; ; a Guide to Implementation 
of the Pre-service Program; and a Guide to Implementation of the Cnntinninn 
Education Program . 
Anticipated Accomplishments 

Instructional resources and the Engineered Internship . By April 1, 
1973, of the 37 modules to be developed, five will be in the planning for 
development phase, five in prototype development, five in prototype form, 
eighteen in revision for field form, one in field testing and three in 
revision based on field testing. (See Figure 4, p. 72). Competence 
assessment instruments for four modules will be available in initial form 
ready for prototype test and instruments for two modules will be in 
operational form. 

By April 1, 1973, information collected on the Fall 1972 performance 
of the Engineered Internship will have been analyzed, and this infonnation 
will be used to revise the Engineered Internship. 

By April 1, 1973, a cost analysis will have been completed covering 
the development and validation of all projected training materials and 
competence assessment batteries. Based on this analysis, a detertiii nation 
will be ma de as to the feasibility of development of all or parts of the 
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Analysis and Definiti on Series and Management Series and their 
related assessment instruments. By May 31 , 1973, 32 modules will be in 
the Instructional System; along with externally developed alternate or 
supplementary resources (at least 25). The modules in the system at this 
time will represent at least seven of the nine competence areas (including 
tJie Educational Information Consultant series) and will constitute 32 of the 
37 modules planned. Of these, eight will be in prototype form, 11 under 
revision for field form, 10 under field testing and three in 
operational form. Competence assessment Instruments for two modules will be 
av.ailable in operational form, instruments for four more -modules will be in 
prototype testing, and those for six additional modules will have been 

I 

developed in initial form ready for prototype testing. 

If the decision Is made in April to proceed with the Analysis and 
Definition Series and the Management Series, their advanced designed and 
developmental planning will have been completed. 

By August 31, 1974, procedures, instructional resources, and assess- 
ment instruments for the Instructional System will all have been developed 
and validated and will be ready for operational use and dissemination to 
rneat Fall 1974 training requirements. 

Guid e to the Instructional System . The purpose of this guide is to 
present a one-volume, comprehensive description of the Instructional System 
for use both as an orientation to the system and as an operational guide 
for participants. The guide, an adaptation and extension of the current 
Instructional Resource Manager's Guide , will include sections describing 
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in detail the underlying rationale; the operational, plan of the system; 
definitions of roles and functions of participants; and their auxiliary aids, 
such as the Catalog of Instructional Resources and the Guide to the 
Competence Assessment System (described on page 66.) It will include a 
special section describing methods for its installation. 

By April 1, 1973, the prototype form of the Guide to the Instructional 
System will have been developed and will be undergoing testing with trainees, 
Instructional Resource Managers and work supervisors. 

By May 31, 1973, the guide will be nearing the end of Spring 1973 
prototype testing. Throughout the testing period continuous feedback on 
the adequacy, clarity and usefulness of this volume will be solicited from 
trainees, Instructional Resource Managers, work supervisors, and others so 
revision of the guide- can be undertaken during the summer of 1973. 

By August 31, 1974, the Guide to the Instructional System will have 
undergone prototype testing during Spring 1973, and revision based on feed- 
back from prototype testing will have been accomplished during Summer 1973. 
Field testing will have taken place in Fall 1973 and Spring 1974. Revision 
based on field testing and preparation of the operational form will be 
accomplished. 

Catalog of In-jtruc t ional Resources . This catalog will be developed 
for use by the Instructional Resource Manager and trainees. It will 
list and describe all modules, supplementary readings, books, journal 
articles, etc., that constitute the instructional resources. It will 
also include instructional materials developed elsev/here but adapted 
for use in the Instructional System. 
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At the present time, the catalog is part of the Instructional 
Resource Manager's Guide . By April 1, 1973, it will be produced as a 
separate document, accounting for all the instructional resources 
available in the system at that time. The catalog will he ready for 
dissemination by August 1974. 

Guide to the Implementation of the Pre-service Program . Installation 
and operation of pre-service and continuing education programs in training 
institutions and agencies are guided by two implementation models, the 
Pre-service Program Model and the Continuing Education Program Model. 

The pre-service Program Model is degree-oriented, set in a college 
or university with a graduate program willing to sponsor eduational DD&E 
as an area of specialization. Students accepted into the program will 
already hold Bachelor's degrees and will have the MA as their goal. Once 
accepted for the program, trainees will be placed in internships at DD&E 
agencies or other suitable work settings. They will be assigned to an 
Instructional Resource Manager who will assist them and their work supervisor 
in planning internship activities as assigning pertinent materials from 
the Instructional System. (A degree program based on the same general 
.model could be implemented which would grant BA degrees in Educational 
Development. ) 

The approach to implementing the Pre-service Program will be presented 
in a guide containing the following items: 

a. The Design of the Entry-Professional Functional 
Competence Training System . This section will 
provide a complete description of the Functional Competence 
Training System and an orientation to its organizing 
concepts and essential characteristics. 
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How to Insta ll a Pre-service Program. This section will 
describe procedures for obtaining accreditation for programs, 
instructional staff, physical facilities, and instructional 
resources. 

The Guide to the Instructional System . Described earlier 
on page 60. 

^- The Catalog of Instructional Resources . Described earlier 
on page 61. 

e. Stud ent Personnel Services . This section will include 
procedures and tools needed to recruit, select, orient, 
guide, place and follow-up trainees in the program. 

f. The Employm ent System . This section will present a 
description of suggested procedures for establishing 
cooperative relationships with employing agencies, 
facilitating establishment of internship positions and 
job placement of trainees. 

9- The Guide to the Comp etence Assessment System . Described 
on page 66. 

^- The Catalog of Competence Assessment Instruments . . Described 
on page 66. 

^' The Quality Control of the Instructional System and the 
Com petence Assessment System . Described on pages 67 and 68. 
By April 1, 1973, specifications will be formulated for the guide 
and a narrative outline form will be developed. By May 31, 1973, the 
following sections will be developed in draft form: How to Install the 
Pre-service Program; The Instructional System; the Catalog of Instructional 
Resources; the Competence Assessment System; the Catalog of Competence 
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Assessment Instruments. By August 31, 1974, theguide will be fully 
developed and validated by testing with at least two users during Fall 
1973 and Spring 1974, and with at least three potential users during 
the same period. 

Guide to Implementation of the Continuing Education Program . The 
setting for the in-service or Continuing Education Program is an R&D 
agency with employees who require training incompetences covered 
by the Functional Competence Training System. The number of modules 
used and whether or not credit is awarded will depend on the needs of 
the individual student and the availability of an accrediting institution. 
The approach to implementing the Contini;ing Education program is presented 
in a guide which provides detailed information on procedures to follow 
and resources required to install and operate the Continuing Education 
Model in an R&D agency. 

The content of this guide will be similar to the Guide to the Implemen- 
tation of the Pre-serv ice Program . However, section (b) How to 
Install the Continuing Education Program', incorporates information presented 
in sections (e) and (f) of the pre-service guide, describing procedures 
for obtaining accreditation for programs, instructional resources 
management capability, physical facilities, instructional resources, and 
selection and piacemeiit of trainees. 

By April 1, 1973, specifications will be formulated for theguide 
and a narrative outline will be available. By May 31, 1973, the following 
sections will be developed in draft form: How to Install the Continuing 
Education Program; Guide to the Instructional System; Catalog of Instructional 
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Resources; Guide to the Competence Assessment System; Catalog of 
Competence Assessment Instruments, "y August 31 , 1974, the Guide will 
be fully developed and validated with at least two user organizations 
and two potential user organizations during Fall 1973 and Spring 1974. 
B. The Competence Assessment System 

The Competence Assessment System provides individuals, R&D agencies 
and training institutions with a method for assessing competence in 
the technical areas of educational DD&E. The system delineates DD&E 
competences by level of proficiency, provides methods for assessing 
competence intrainees and suggests criteria for certifying competence. 
A validated system of competence assessment has a utility far beyond 
use with the Instructional System. It can become an instrument that 
employers can use to screen applicants, to place new employees, to 
assign employees to tasks, to identify employee competence deficiencies 
and to plan a personnel development program. 

DD&E competences are organized according to competence sets, 
subsets and elements. The competence subsets generally correspond to 
the content of modules in the Instructional' System and the competence 
elements to learning episodes within modules. Competence assessment 
batteries will consist of a number of assessment techniques for each 
competence subset A description of the components of the competence 
assessment batteries, as well as the status of development at various 
time points, will be presented in Part Two: Product Development . The 
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Competence Assessment System consists of competence assessment batteries 
(developed on a modular basis), a handbook of procedures for using them, 
the Guide to the Competenc e Assessment System , and a Catalog of Competence 
Assessment Instruments . 
Anticipated Accomplishments ; 

Guide to t he Competence Assessment System . This handbook will- 
specify procedures used to assess trainee competence based upon knowledge 
acquired in the competence subsets -(through the modules) and application 
of this knowledge in the Engineered Internship. Specifically, it will 
give detailed instructions to the Instructional Resource Manager in the 
use of the competence assessment batteries, criteria against which 
trainee performance can be evaluated, and methods for competence 
certification. Procedures will also be specified by which a trainee 
can "challenge" certain parts of the training and/or be certified in 
competencies without completing the training. 

Catalog of Competence Assessment Instruments . This catalog will 
be developed for use with the Instructional System. It will describe 
the assessment instruments associated with the competence .set£ and 
subsets. 

By April 1, 1973, a prototype form of the Competence Assessment 
System and the guide will be developed and undergoing testing. The 
first form of the catalog will be developed and produced as an independent 
document. By May 31 , 1973, the Competence Assessment System will be 
nearing completion of its prototype test, and revisions in its design 
will begin, based on feedback from the prototype testing. The catalog 
will be up-dated by September, 1973. By August 31, J974, Vhe Competence 



Assessment System, the guide and the catalog will have undergone prototype 
testing, revision, field testing and revision, and the operational fonns 
will have been developed as an integral part of the Functional Competence 
Training System and be available for dissemination. 
C. The Quality Control S ystem 

The Quality Control System provides means and methods by which 
implementing agencies may evaluate the adequacy and maintain quality 
control of use of the Instructional and Competence Asses .ment Systems. 
Quality control instruments will be developed and a guide provided for 
each system, including what questions to ask", what information and 
data are to be collected with the use of what instruments, how to 
analyze the infonnation and data collected, and how to interpret them. 
Anticipated Accomplishmen ts : 

Guide to the Quality Control of the Instructional System . By 
April 1, 1973, an initial draft of portions of this guide will be 
developed and tested with participating faculty at CSUSF and Far West 
Laboratory staff. This draft will specify procedures by which an 
implementing agency may continuously monitor implementation of the 
training and the Engineered Internship. By May 31, 1973, revision of the 
initial draft will be undertaken, based on testing. The remaining 
portions of theguide will be in draft form, and a review of each 
portion will begin as it is completed. This guide will be tested with 
the institutions using the system during Fall 1973 and Spring 1974. 
By August 31, 1974, an operational form will be completed, becoming 
an integral component of the Functional Competence Training System. 
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Guide to the Quality C ontrol of the Competence Assessment Sy stem. Th 
guide will outline methods for collectin? information/data on application 
of the system and procedures for modifying and up-dating it. By April 1, 
1973, the initial draft will be complete. By May 31, 1973, revision of 
the draft will be in progress, based on review by participating faculty 
at CSUSF and Far West Laboratory staff. Planning for implementation of 
the procedures in this guide will begin during field testing of tne 
Instructional System in Fall 1973. Information will be collected on the 
performance of the guide from professionals involved in the prototype 
implementation of the pre-service and continuing education models. The 
guide will be revised based on an analysis of this information. By 
August 31, 1974, an operational form will be complete and become an 
integral component of the Functional Competence Training System. 

Descriptions of the instruments used in the Competence Assessment 
System and timetables for their development will be found in Task Two; 
Product Development , pages 75 through 79. 
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PART TWO: PRODUCT DEVELOPMENT 

Product development involves the specification, selection for design, 
formative development, field- testing, evaluation and revision of the 
materials and instruments that provide the required resources of the Instruc- 
tional System and the Competence Assessment System. Scope of work will be 
reported for t^/o areas: the development of instructional resources and 
the development of competence assessment instruments. 

A. D evelopment and Adoption of Instructional Resources 
The Instructional System uses tv/o kinds of resources: materials 
developed by the Consortium and materials adopted/adapted by the Consortium. 
Both kinds of resources are used by the trainee and the Instructional Resource 
Manager. 

Materials Development 

Instructional materials will be developed at the entry professional 
level in eight competence areas and are available in a ninth/ The 
instructional materials for each competence area are organized into 
senes.: (a) Planning and Design, (b) Information/Data Collection, (c) 
Communication Skills, (d) Developmental Engineering, (e) Evaluation, (f) 
Analysis and Definition (g) Dissemination and Marketing, (h) Management^, and 
(i) Educational Information Consultant. Within each series are sub-divisions 
called m_odules_, each of which focuses on a cluster of competence elements. 
Modules are referred to by numbers within their series- for instance. 
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^I. ■ K 1" ^^"catio ji. Volumes One and Two, Ber keley, California - 

te?m'" Lrs1"'Jse7in'°fh."S"-'°";^ ""r"''' Development, 1970 \he ' 
term course used in the Design Report is now replaced with the term "series". 

^ Three modules have been, adopt- J from the Educational Information 
^ consultant Instructional System described in Appendix L. 
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Module 3^ is the second module of Series 3, Communication Skills. Within 
modules learning activities are grouped by episodes which attend to 
specific competence or skills within the set. Each episode has its own 
objectives, learning activities , and progress checkpoints that allow the 
trainee. Instructional Resource Manager and work supervisor to evaluate 
the trainee's progress. Typically, modules are designed to be self- 
administered, but one trainee may also work with another trainee or in a 
small group. 

Modules are designed around specific performance objectives derived 
From task analysis. They are designed to enable the learner to achieve a 
p re-determined level of competence. (A definition of levels of competence 
appears in Appendix G) In compliance with the suggestions of the Researcher 
Training TasK horce, Attachment Two outlines the instructional content in 
each competence area and describes the format of the materials. Exemplary 
competence assessment instruments are also enclosed as Attachment One. 

A schedule for development of instructional resources from January 1, 
1973, through August 31, 1974, is presented on page 72. In that schedule, 
the 37 modules now comprising the resources for DD&E training at the entry 
professional leveT are identified. It also groups the many specific 
developmental tasks into seven phases: 

Phase 1. Prepare developmental plan for series, 
assign competence, outling content. 

Phase 2. Prepare prototype form. 

Phase 3. Test prototype form. 

Phase 4. Prepare field test form. 
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Phase 5. Test field test form. 

Phase 6. Prepare operational form. 

Phase 7. Disseminate operational form. 
Each module is designed in relation to others in a series. In several 
modules relationships across series have been incorporated, eg. "Planning 
for Evaluation" (Module 1.4 in the Planning and Design Series) is also an 
integral part of the Evaluation Series. Phase 1 covers this design and 
organizational effort. It includes the allocation of competence identified 
Tn the initial Design Report and the Competence Matrix analysis and an outline 
of content developed by subject-matter specialists, author(s), and product 
development. staff. In this phase authors receive a Module Developer's Kit 
which provides specific instructions and performance criteria for each facet 
of development. 

Phased of the production effort places the prototype form of the module 
in the hands of students and reviewers. During development of the prototype 
form, as parts of a module are developed, they are tried out with two or 
three trainees or subjects representative of the trainee population. 

Phase 3 provides for prototype testing of the module with the involvement 
of five or more trainees, and includes such additional evaluative activities 
as reviews by content specialists, developmental staff and work supervisors. 

Phases 4 and 5 repeat the production and testing cycles. The revised 
module is given to 10 to 20 more users. Their evaluations form the basis for 
the final revision. Phase 6, and the preparation and printing of the operational 
form. Phase 7. 
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We present below a summary statement of the status of module develop- 
ment at the "milestone" dates. The Schedule of Developmental Activities, 
Figure 4, gives the phase-of-development timeline for each of the 37 
modules . 



April 1, 1973 

In Phase 1 (Development of Module Plans) 
Overview Module 

Analysis & Definition Series: 6.1, 6.2 
Management Series: 8.1, 8.2 

In Phase 2 (Preparation of Prototype Form) 

Evaluation Series: 5.3, 5.4, 5.5 
Dissemination/Marketing Series: 7.2, 7.3 

In Phase 3 (Prototype Testing) 

Developmental Engineering Series: 4.1, 4.3, 4.4, and 4.5 
Dissemination/Marketing, 7.1 

In Phase 4 (Field Test Form Preparation) 

Planning & Design Series: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6 
Information/Data Collection Series: 2.2, 2.3, 2.4, 2.5, 2. 
Communication Series: 3.1, 3.2, 3.3, 3.4 
Developmental Engineering Series: 4.1 
.Evaluation Series: 5.1, 5.2 

In Phase 5 (Field Testing) 

Information/Data Collection Series: 2.1 

In Phase 6 (Preparation of Final Form) 

The three modules adopted from the EIC instructional system 



•May 31, 1973 

In Phase 2 (Preparation of Prototype Form) 
Overview Module 

and, contingent on April 1 cost analysis 
Analysis & Definition Series: 6.1, 6.2 
Management Series: 8.1, 8.2 

In Phase 3 (Prototype Testing) 

Evaluation Series: 5.3, 5.4, 5.5 
Dissemination/Marketing Series: 7.2, 7.3 
Developmental Engineering: 4.1, 4.3, 4.4 



In Phase 4 (Field Test Form Preparation) 

Planning and Design Series: 1.6 
Information/Data Collection Series: 2.2, 2.4 
Coimiunication Skills Series: 3.1, 3.3, 3.4 
Developmental Engineering Series: 4.2, 4.5 
Evaluation Series: 5.1., 5.2 
Dissemination and Marketing Series: 7.1 

In Phase 5 (Field Testing) 

Planning and Design Series : 1.1, 1.2, 1.3, 1.4, 1 5 
Information/Data Collection Series: 2.1, 2.3, 2.5. 2 6 
Coimiunication Skills Series: 3.2 

In Phase 6 (Preparation of Final- Form) 

Educational Information Consultant Series: 9.1, 9.2, 9.3 

By August 31, 1974, all modules will be in operational form, ready for 
dissemination. The completion dates will, of course, fall over a span of 
time. Three modules are projected as ready in operational form by 
September 1, 1973; twelve more by January 1, 1974; another ten by April 1, 
1974; twelve by July 1, 1974; and the remaining modules completed between 
July 1 and August 31 , 1974. 
Materials Adoption/Adaptation 

Instructional materials adopted from external sources fall into four 
categories: (a) those materials that, in their present state, clearly 
satisfy Laboratory design requirements with respect to competence set, 
instructional method, instructional level (entry-professional), and 
feasibility considerations such as cost, packaging, and copyright 
stipulations; (b) those materials that, with suitable modification or 
further development, can satisfy requirements specified in (a) above; 
(c) those materials included within the scope of materials of a particular 
module; and (d) those reference materials that are used as outside readings 
serving to enchance or expand the scope of the module but not requisite to 
its successful completion. 
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Anticipated Accomplishments 

By April 1, 1973, all modules under development will have been 
reviewed with a view toward identifying externally developed resources — 
fitting one of the categories defined above -- most relevant to them. By 
August 31, 1974, all materials collected or identified will be listed in 
the Catalog of Instructional Resources under the modules to which they 
are related. Externally developed materials which are integral parts of 
the system will be ready for dissemination; other optional or supplementary 
material will be referenced in the appropriate module and listed in the 
Catalog of Instructional Resources. 

B. Development of Competence Assessment Instruments 
For each DD&E competence set competence assessment instruments will 
be developed. As presentely designed, these instruments will consist of 
the following devices: 

1- Trainee Self-Rating Scale . This instrument will be completed 
by the trainee prior to the assignment of instructional material and again 
at the conclusion of the trainee's instruction and application phase of 
training. The purpose of this scale is to obtain the trainee's perception 
of his/her proficiency in performing tasks related to the various competence 
elements at the beginning and end of training in a particular area. The 
pre-instruction rating may be used along with other indices to judge 
whether or not the trainee has sufficient proficiency in a particular 
competence; subset to warrant challenging it (i.e., credit by examination). 

2. Supervisor Rating Scale . This instrument, similar in form to 
the Trainee Self-Rating Scale, is designed to obtain the work supervisor's 
perception of the trainee's level of proficiency in the relevant competencies. 
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This assessment of the trainee's skill and knowledge will also be obtained 
before and after instruction and application. 

3. Job Knowledge Test. This i nstrument wil 1 test the trainee's 
job knowledge relative to the particular competence subset. It will be 
used as a pre-test to determine the trainee's knowledge base prior to 
training and will be used after training to assess knowledge gain. This 
instrument, along with others, may be used by trainees wishing to 
"challenge" a competence subset. 

4j Simulation Exercises . Miniature job tests and more complex 

simulations require the trainee to apply his/her knowledge to a. simulated 
situation or miniature job sample typically encountered in DD&E work. These 
exercises, as well as the related Job Knowledge Test, will be calibrated by 
testing a sample of professionals operating at target level (at least MA 
and one year experience or BA and two years experience). Technical 
review by work supervisors of the simulations and Job Knowledge Test 
and of the test protocols will be used to establish content validity. 

5. Product Rating Scale . This scale is to be used to evaluate 
products developed on the job by trainees, a rating scale to indicate 
the quality of such products will be developed and calibrated through use 
in rating products developed by professionals operating at the target 
level and by involving work supervisors in the development and calibration 
of the rating scale. 

Standard procedures for test development and validation will be 
followed as far as practical, given the re'iatively small sizes of the 
trainee groups and the target level calibration groups (20 to 40 persons 
in each group). 
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Steps in development of competence assessment instruments : 
Step 1: Prepare draft 

Step 2: Administer to three pairs (see below) 

Step 3: Revise, based on Step 2 

Step 4: Administer to three other pairs 

Step 5: Revise, based on an analysis of findings accumulated 
during Step 4 (Note: Steps 4 and 5 may have to be 
repeated several times to achieve a satisfactory 
set of instruments) 

Step 6: Administer to ten pairs 

Step 7: Revise, based on analysis of findings accumulated 
during Step 6 (Note: Steps 6 and 7 constitute a 
preliminary calibration test. Although the sample 
is quite small, it should be large enough to spot 
operational problems and to afford very rough 
statistical estimates.) 

Step 8: Administer to at least 30 additional pairs 

Step 9: Use the information gathered for the purpose of 

calibrating the instruments, setting standards, and 
documenting face validity 

Step 10: Revise for operational form 
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Pairs refer to an employee (or trainee) and his work supervisor, m 
steps 2 and 4 one pair each will be drawn from three kinds of population: 
(1) the incoming pre-service student, (2) students who have completed 
relevant training, and (3) DD&E agency personnel with MA and at least one 
year experience or BA and. at least two years' experience. The last group 
constitutes the "calibration" population. In Steps 6 and 8, all pairs are 
drawn from the calibration population. Note that the battery emerging ' 
after Step 10 will be a "validated" set of instruments that will be usea 
for definitive pre-training, post-training comparison, in order that the 
trainees and the instructional system may in turn be assessed. 
Anticipated Accomplishments 

By April 1, 1973, two batteries will have been "crash" developed 
through Steps 1 through 10 above in order to validate the instrument 
development and calibration procedures and to provide a realistic basis 
for cost and time estimates for subsequent battery development. We 
intend to provide technical specifications for the development and vali- 
dation of competence assessment instruments, including such aspects as 
minimum number of persons and characteristics of calibration and validation 
groups, level in inter rater reliability, level of internal consistency 
reliability, standard errors of measurement, plan for analysis, etc. 
In addition, at least four more sets of assessment ins.ruments (corre- 
sponding to .our instructional modules) will be in draft form (Step 1) 
■ready for development following these technical specifications.' 
By May 31, 1973, prototype testing of these four sets will be 
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nearing completion (Step 4 or 5) and initial fonns (Step 1 or 2) will be 
available for six more sets. By August 31, 1974, competence assessment 
batteries for all the series will have been developed", calibrated, validated, 
and prepared in operational t^rm to be included in the Functional Competence 
Training System. 
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PART THREE: TRAINING IMPLEMENTATION 
A. THE PRE-SERVICE PROGRAM 

_ A pre-service training program leading to the MA degrea in education 
With a concentration in DD&E is currently in operation at CSUSF. . A Program 
Coordinator handles the administrative aspects of program implementation 
and operations and arranges internship positions for trainees. Thirty-one 
trainees a^e currently enrolled in this program-14 Cycle I trainees (entered 
program in Fall 1971) and 17 Cycle II trainees (entered program in Fall 1972). 
All trainees have been placed in' internships, either with school districts 
( 14). R&D agencies (15 ). or industrial training establishments (2 ). 
All trainees have been assigned to one of three Instructional Resource 
Managers who assist them and their work supervisors in formulating work plans, 
arranging study plans, and providing feedback on trainee work. Trainees 
may select modules now developed and/or other relevant training materials 
to satisfy their competence training requirements. It is planned that 12 
new trainees (Cycle III) will be recruited for the Fall 1973 program at CSUSF. 
A pre-service program similar to the one at CSUSF will be launched 
during the school year 1973-74 at another institution outside the geo- 
graphic region of the Far West Consortium in order to accomplish a pre- 
liminary (operational) test of the program's transportability. Instructional 
resources and guidebooks needed to implement this program will-be provided 
by the Far West Consortium. 
Anticipated Accomplishments 

By April 1. 1973. Cycles I & II will be in progress in the Far West 
Consortium region, and initial plans for implementation outside the region 
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will be developed. By May 31, 1973, preparations for Cycle III in the Far 
West Consortium region will begin, and negotiations will be underway for 
introduction of the pre-service program at an institution outside ^,he Far 
West Consortium region. 
B. THE CONTINUING EDUCATION PROGRAM 

Continuing education programs to meet the training requirements of 
R&D agency employees will be implemented during 1973-74. At least one 
program will be initiated at an R&D agency within the Far West Consortium's 
region and at least one at an agency located elsewhere. These programs 
may be accredited through training institutions or may be non-accredited. 
Instructional resources and guidebooks needed for implementation on a 
prototype test basis will be provided by the Far West Consortium. At 
least six students will be enrolled in each of these programs. 
Anticipated Accomplishments 

By April 1, 1973, initial plans will be developed for implementation 
of the program irj both the Far West Consortium region and another location. 
By May 31, 1973, negotiations will be underway for a 1973-74 introduction 
of the program in both areas. 
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PART FOUR: DISSEMINATION/UTILIZATION 
Dissemination/utilization involves preparation of systems and 
materials developed by the Consortium in an operational, transportable 
form and arrangements for the use of those systems and materials by 
training institutions and R&D agencies outside the region of the 
Consortium. 

Anticipated Accomplishments 

By late February, 1973, a preliminary dissemination plan will be 
prepared. At the AERA meeting in February, this preliminary plan will 
be critiqued with three or four potential user institutions [training 
institutions and R&D agencies). These preliminary discussions will 
cover both pre-service and continuing education implementation. The 
outcome of these discussions will be documented by April 1, 1973. 

By May 31, 1973, an analysis of the dissemination/utilization problem 
will be developed. Dissemination requirements will be formulated and at 
least two approaches to dissemination will have been specified. A document 
will report the outcome of the analysis and describe and evaluate the 
dissemination approaches. Negotiations will be underway to arrange for 
pilot implementation of at least one pre-service and one continuing 
education program outside the Far West Consortium's geographic region. 

By August 31, 1974, the two models and their dissemination approaches 
will have been empirically tested. System descriptions, guidebooks, and 
instructional and assessment materials will be developed in operational 
form and ready for release. Arrangements will be made for the implementation 
of the system outside the boundaries of the Consortium's geographic region. 
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PART FIVE: PROJECT MANAGEMENT 
Project Management involves the coordination of the affairs of the 
Consortium and the management of the .in-house staff assigned to the DD&E 
project. 

Membership in the Consortium will be reviewed in order to retain as 
members only those agencies which continue to be functionally involved in 
the project. The Consortium Board will pay special attention to the 
development of a continuing education program and to long-range planning 
^^or pre-service training and continuing education beyond 1974. These 
products are anticipated here: 

1. A report on a "pi an for continuing education program for 
1973-74 (available by May 31, 1973). 

2. An initial statement of the long-range plans beyond 1974 
(available by May 31, 1973). 

3. A detailed plan and specific arrangements for continuation of 
training in the Consortium geographic area beyond 1974 
(available by late 1973). 

The in-house :staff is organized into four functional components: 
(a) systems development, (b) development of instructional resources, 
(c) development of competence assessment instruments, and (d) field 
services and dissemination. A senior staff member coordinates each of 
the components. P-oject management is carried out by a management team 
consisting of the Principal Investigator, the Program Director, and the 
componert heads. Products associated with project management include 
interim reports and final reports to be submitted to NIE. 



SUMMARY STATEMENT OF OUTCOMES 

By August, 1974, the Consortium will deliver: 

1- Pre-service and Continuing Education Models for the training of 
entry professionals in educational DD&E. 

2. Development products related to the program as follows: 
° A Guide to the Instructional System 

° A Guide to the Quality Control of the Instructional System 
° A Guide to Pre-Service Program Implementation 
° A Guide to Continuing Education Program Implementation 
° Over 30 instructional modules developed by the Consortium 
° A Catalog of Instructional Resources 

Competence assessment instruments for each training module 
° A Catalog of Competence Assessment Instruments " 
° A Guide to the Competence Assessment System 
° A Guide to Quality Control of the Competence Assessment System. 

3. Knowledge products to include findings relevant to: 

° the design of a functional competence training system 
° the development and validation of competence-based 

training programs and instructional resources 
° development and validation of the Engineered Internship 

and a study of variables impinging upon it 
° development and validation of fJ^e-service and continuing 

education programs 
° advantages and disadvantages of a training development consortium 
°' validation of needed skills in educational DD&E 
° new patterns for synthesizing the "academic" and "work" 



environments in training 

an individualized approach to responding, in economic 
and timely ways, to changing needs for on-the-job 
training and upgrading of DD&E personnel. 
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Berkeley, California 
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California State University at San Francisco 
San Francisco, California 

Canada College, 
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Local Education Agencies 

(represented by Dr. Paul Ehret, San Lorenzo School System) 
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■Appendix B: RATIONALE 

The basic purpose of the Far West Consortium is to develop and 
validate a transportable training system for professionals who are 
already employed, or who will move into future positions in the field 
of educational development, disseminatjon, and evaluation (DD&E), with 
initial emphasis on the requirements of che San Francisco Bay Area 
in the early 1970's and for the entire country thereafter. 

There are too few appropriately trained personnel to carry out 
the development, dissemination, and evaluation of educational products 
and processes. This deficiency can be corrected by a training program 
that can produce qualified DD&E personnel and upgrade the skills of 
people already working in the field. 

Hendrik Gideonse stated in 1969: "Educational research in the 
United States is going through a period of agitation, ferment, and 
perhaps even crisis." His words apply today — not just in the field 
of educational research, but even more in the emerging discipline of 
educational DD&E. Many R&D institutions established with USOE funding 
in the early '60's represent a federal commitment to educational renewal 
Trained personnel were scarce in the early '60's, so new agencies drew 
most of their senior professional staff from university research 
communities. .Most of the training and experience of thvs group, 
however, was relevant to conclusion-oriented research rather than the 
decision-oriented, disciplined inquiry of development, dissemination, 
c.nd evaluation. Furthermore, literally no formally trained personnel 
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were available for middle-level staff requirements, since virtually 
no demand for their services had existed. ^ 
From the beginnings of the post-Sputnik federal support.-.of science 
and language cur^culum refonns through the enactment of ESEA legis^tion 
in 1965, a new demand arose for men and women qualified to undertake 
professional tasks in educational DD&E. In a 1969 study, Clark and 
Hopkins projected a probabl.; five-fold increase in the number of R&D 
posirons in the ten year period ending in 1974. Their minimum growth 
assumptions had projected a decline in research positions from 95.6% 
of the total R&D positions to 38% in 1974, whereas d evelopment positions 
had been seen as increasing to 45% of the 1974 total. This analysis of 
manpower requirements, along with our own of the San Francisco Bay Area, 
led us to place our initial emphasis on pre-service training programs 
in educational DO&E. 

John Hopkins (1971) updated th- 1969 Clark and Hopkins study, 
indicating that subsequent econo;iiic and political shifts have occurred 
which make the original projections too high. The earlier study was a 
financially-based projection 6f demand for trained personnel. The update 
was based on actual funding through 1971 and projected funding through 
1974. The update projected a need for 8,699 positions by 1974 and indicated 
that most of these may already be filled. (See Table 1 .) The reasons for 
these changes in the projections are: (1) economic factors indicating 
very little expansion in the number of positions available since .1966, and 
(2) nomarked increases projected in funding supporting new positions. 
These changes clearly indicate a shift from an expansionary labor market 



Table 1 

Projected 1974 Positions in Educationlfi' -RD^r 



Sub-Units 
(Federally Supported) 


Projected 
Position 


pother Settings 


Projected 
Positions 


Regular Projects 
Laboratori ^ 


991 
564 


Schools and colleges 
of education 


1,244 


Title III Centers 6% 


469 


Other behavioral and 
social science departments 


527 


State Educational 
Agency Res. Units 


361 


Schools and departments of 
Psychology 


500 


Small Projects 
R and D Centers 


354 
307 


Other discipline and 
academic departments 


491 


NSF Course Content 
Project 


216 


State Departments 
of Education 


457 


Handicapped Materials 
Centers 


^ 193 


Business and industrial 
organizations 


300 


Clearinghouse 


180 ' 


Private research institutes 
and agencies 


300 


Vocational Education 
Research Coordinating Units 


177 


Schools and school systems 


270 


Handicapped R&D Centers 


127 


College andunlversity . 
administrative units 


205 


Vocational Education 
R&D Centers 


88 


U*S. Office of Education 


156 


Early Childhood Centers 


39 


Professional associations 


90 


Policy Study Centers 


13 


Inter-agency Associations 


50 


Total Positions 
Federally Supported 


4,079 


"otal Positions Supported 
from other Sources 


4,590 


Final Projection of Positions 

Estimated Research Positions 
Estimated Development Positions 
Estimated Diffusion Positions 


8,669 
^2,861 
^4,334 
^1 ,474 



^Figures bused on proportions projected in the original Clark and 
Hopkins study (1969, p. 288) of: research, 33%; development, 50%; and 
diffusion, 17%. 
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to a relatively stable one in terms of total numbers. However, the updated 
positions confirm shifts in demand for trained personnel (generally away 
from research and toward greater relative demand for developers and 
evaluators). 

Educational DD&E personnel are now recruited from a variety of 
disciplines, but there is relatively little reliable information available 
on their previous work experiences (Schalock, et.al., 1972') J A regional 
survey by Hood and Banathy (1970) suggests that there may be substantial 
competition from business and industry for personnel with the same general 
skills as those required in educational R&D. Hence we can foresee a need 
to meet requirements for trained personnel from various competing sectors 
as well as the necessity of training and continuing the educu^ion of 
personnel in educational R&D. The studies of Brickell (1970, 1972 in 
pir^eparation), Byers ' (1971 ) , Fleury , Cappel luzzo and Wolf (1970), Sanders 
and Worthen (1970), and York (1968) indicate that training in development 
and diffusion (as well as in educational programrr.ing and decision-making) 
. may still be quite inadequate. Numerous studies revealing the low 
quality of s ^ i ESEA Title I and Title III projects as well as other 
school-based exemplary programs (Wargo, et.al., 1971) can be seen as 
further danger signals if improvement of educational practice calls for 
the significant improvement of disciplined inquiry skills in operating 
educational agencies. 

1 

The Oregon Studies (Schalock, e.a"!., 1972, Volume I, Chapter 6) provides 
our best current source of information, but is based on data provided by 
approximately 100 persons in 15 carefully selected DD&E projects. 



Even in an era of reduced federal spending for educational R&D, 
both manpower surveys and the day-to-day realities of life in development 
agencies reveal a need for middle-level professional personnel.^ 
Adequately trained development and diffusion personnel cannot be found in 
sufficient numbers to effect educational change in the 1970's. For example 
there is no credible evidence that most of the "principal investigators" 
have learned educational DD&E other than the "hard" (and costly) way, or 
that these DD&E managers iiave either tpe time or resources to 
efficiently train those they supervise. The applied character of 
development and installation tasks (as contrasted with the doctoral - 
level training provided for the university researcher) has clearly 
shown a need for intensive inservice training programs in mission 
oriented R&D agencies and in allied industrial firms. To date, there 
is no really reliable information regarding even the proportions or 
numbers in the functional areas of DD&E, but available information 
(Hood and" Chorness, 1972, pp. 3.5, 3.6; Shalock et. al . , 1972, 
pp. 87-93) suggests that continuing education will be essential to 
meet the changing structure of work requirements and to upgrade staff 
competence in "directed" and "applied" R&D programs. 

The educational P.&D community has often been reproached by the 
legislative and exicntive branches of government for its less-than- 
hoped-for productivity and its less-than-universal impact on the 
improvement of educational practices. Pragmatic evidence of impact in 

2 

Ibid. The Oregon Studies data show that over three- fourths of the 
personnel in the 15 DD&E projects held MA or BA degrees, and lesb than 
a fourth held doctorates. 
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the improvement of educational practices may be more eviaent only after 
a well -trained cadre of development, dissemination, and evaluation 
personnel gains experience in the R&D agencies which have assumed the 
major burdens of creating and validating educational alternatives. 

The Rationale for a Consortium Approach 

Clark and Hopkins (1969, pp, 422-485) point out the need for a 
strategy and outline a series of tactics for meeting educational R&D 
manpower requirements. A major short-term objective, in their view, 
would be establishment of a trairung network to produce development and 
diffusion personnel in large numbers in a relatively short period (p. 426). 
They suggested several tactics pursuant to this objective, including the 
establishment and support of experimental or developmental training 
programs, the initiation of course content improvement programs, and the 
establishment of consortia of institutions for inservice development of 
DD&E personnel in education. 

Taken together, these three reco»nmendations suggested the basis for 
a new pattern combining the strengths of several kinds of agencies. An 
optimum configuration of cooperating institutions, we beTieve, would 
require: 

1. One or more agencies with competence in the design, development, 
evaluation, installation, and maintenance of training systems; 

2. One or more colleges or universities willing to develop and 
provide graduate-level programs for personnel In DD&E work at 
the professional (MA) level; 



3. Two or more large agencies with projects involving substantial 
elements of development, evaluation or diffusion work that can 
provide and support a number of internship positions; 

4. Several agencies which are potential employers of educational 
DD&E personnel and will be able to create positions for 
probationary and permanent personnel; 

5. Representation for educational , community, and student interests. 
Such a combination is desirable since it is highly unlikely that any 

single unit listed above can carry out all aspects of the program. For 
several reasons, it appears that a combination of on- and off-the-job 
training will be more economical that either alone. (See, for example. 
Continuing Education for R&D CarPPr<;. Renck, Kahn and Gardner, 1969 J 
Few educational DD&E agencies are large enough to support, ■nuch less 
develop, the necessary on-the-job courses. Large agencie. capable of 
providing wel 1 -organized internship training positions seem essential. 
Sti.-dents of training research have usually stipulated the necessity for 
training where research is being done (Buswell, 1966). Empirical 
studies of the effect of research assistantships on productivity in the 
field of education (Buswell, 1960; Worthen, 1968) also call attention to 
this point. 

Experience with the Consortium Approach 

Our experiences with the consortium approach during the last two 
years have helped us to gain some new and valuable insights. 

It, appears that in developing a complex model like the Functional 
Competence Training System one should start out with a smaller number 
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of core agencies, A longer lead time than was available to us is also 
required to construct clearly defined developmental models, procedures, 
guides, etc., so member agencies will have detailed guidance with adequate 
examples to ensure consistency and quali;.y of development. Once such models 
procedures, and guides are developed, membership may be expanded if 
furlding is adequate. In our case, a large initial membership also 
created difficulties in providing active and meaningful participation of 
all members, increasing the demand for the limited shared resources 
available, and creating problems in coordination, monitoring, and quality 
control . 

On the other hand, the active participation of experienced R&D 
agencies in the development of the program (such as FWL, AIR, and HumRRO) 
has led to healthy "cross-fertilization" in the forms of learning from each 
other, avoidance of parochial models and views of DD&E, mutual critique. 



involve relatively new and emerging content areas. In well-defined 
content areas there may be less of an advantage and a "single agency" 
model may be more efficient. But when the discipline is emerging, as 
in the case of educational DD&E, the multiple perspective afforded by 
three large experienced institutions such as FWL, AIR, and HumRRO is 
an invaluable corrective for narrowly conceived or unrealistic content. 



ERIC 



13 

Appendix C: DEFINITION OF COMPETENCE AREAS 

Selection of Content .and Structure 

The Design Report indicated tliat when we examined the task analyses and 
compatence inventory survey information, two alternatives for organization of 
training content seemed -.onh ^..amining. 

Processes. The organizing categories of data and people emerged when 
we asked what DD&E personnel work with. Refinement of these categories 
leads to a focus on the skills and knowledges subsumed by such general processes 
as collecting (lata or information), organizing, analyzing, classifying, ordering, 
constructing, synthesizing, communicating, managing, etc. 

Contexts of DD&E phases . Our analytic and retrospective studies of 
the contexts of DD&E work revealed a number r ' fairly distinct kinds of 
activities that tend to occur in cycles. Although the demarcation between these 
phases is vot always clearcut, and there i^ almost always a repetition, of sub- 
cycles of phases within a larger development cycle, we can usually identify most 
or all of the following: 

1. Analysis (systems or operations analysis, including need 

identification and problem definition) 

2. Planning and Design 

3. Developmental Engineering 

4. Evaluation 

5. Dissemination/Marketing 

A mixed approach . These alternative approaches were found to be comple- 
mentary. Training in processes were built into each of the five "context" series 
listed above. But , our analysis indicated a need for separate, supplementary 
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attention to tiiP'^e additional process oriented competence areas: 

6. Cormriuni cation 

7. Information/Data Collection and Organizations 

8. Management 

Functional Context Series 

Analysis and Definition , This series provides the trainee with an ori- 
entation to problem analysis, identification, and problem definition. The trainee 
will practice analyzing needs and identifying problems in a variety of educational 
settings. Besides collecting information on problems and needs from documentary 
information, he will also have experience in developing statements of systems 
requirements and constraints, selection and articulation of design concepts, 
and collecting data on or making a priori judgments regarding alternatives, es- 
timating feasibility, etc. 

Planning and Design . This series provides orientation to and familiari- 
zation with a broad set of competences associated with laying out the overall 
plan for a development or operational solution and the design work of a plan. 
The trainee will formulate goal statements based on a statement of system require- 
ments; derive from goal statements performance specifications; consider or invent 
alternative ways and means by which specifications, can be met; analyze and select 
the most cost/effective alternative; learn ways to present the selected alterna- 
tives; and prepare plans for development, evaluation and dissemination. 

Developmental Engineering . This series provides the trainee with an 
orientation to the wide variety of activities and procedures that may be required 
in fabricating a product or generating a replicable process. Through a series 
of selected developmental case studies, protocols, simulated tasks, and projects. 
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the trainee will learn how to: establish developmental objectives; fabricate 
a simple component and combine components; use test information as a basis for 
modification or improvement; learn to make judgments as to the quality of the 
component under development and suggest revisions; and locate and employ tech- 
nical materials, aids and resources for development. 

Evaluation. The modules in this series provide instructional experiences 
in (a) the logical and methodological basis of evaluation, (b) the kinds of 
evaluation peculiar to DD&E,and (c) techniques and procedures for coping with 
practical evaluation problems. The series will cover the areas of formal and 
informal evaluation. Examples of the former include: examining the test con- 
ditions, reviewing the nature of the instruments used, examining the evaluation 
design, and comparing expected outcomes with actual outcomes. 

The latter area covers the general area of decisions and evaluations 
normally encountered from the inception of a project through to marketing and 
dissemination. Familiarization and some practice with sampling, design, data 
collection, and analysis procedures will be provided as a part of the series. 

Dissemination and Marketing . In this series, the intent is to create an 
awareness of the importance of dissemination, marketing and utilization of 
products and education solutions developed through R&D efforts. The trainee 
will be familiarized with the problems of dissemination, marketing and utili- 
zation and with practical strategies, methods and techniques that can be em- 
ployed in solving those problems. Furthermore, the trainee will acquire an under- 
standing of and practical experience with the use of media, graphics, reproduc- 
tion processes, and communication techniques as they relate directly to dissemi- 
nation and marketing. 
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Process Skills Series 

Information/Data Collection and Organization . The modules of this series 
are designed in two sections, one dealing with collection and organization of 
documentary information and the other dealing with collection and organization of 
quantitative and qualitative data. These two sections can be presented separately 
or as one unit. The information portion will focus on competences in search, 
retrieval and organization of documentary information with emphasis on basic 
library research skills, and proficiency in the use of ERIC, DATRIX, CUE, etc. 
The- data portion will be substantially larger in scope and will focus on providing 
familiarization with commonly encountered methods of obtaining data, and basic 
procedures for reducing, organizing, analyzing and displaying it. 

Communication Skills . This series focuses on receiving, organizing, and 
transmitting information or instruction through oral , written and visual media 
in informal, formal, and technical contexts. The trainee will acquire needed 
competences by means of communication exercises and projects derived from analysis 
of the frequent and critical communication requirements faced by DD&E personnel. 

I 

Additional areas of competence treated in the course include writing and making 
s oral presentations and using visual aids in making such presentations; writing 

review of literature, press-releases and dissemination information; summarizing 
test data; preparing brochures describing projects, activities, products or 
agencies. 

j 

' Management . This series is organized in two basic sections: personnel 

I and operations. The personnel section provides, through role playing, simulated 

situations and experiences in performing supervisory activities. The operations 

! 

^ section deals with management by objectives, work assignments, production scheduling 

> 

I and control, and maintaining communication between teams and within a group. 

ERIC 
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A. Summary of Far West Consortium Personnel and their Functions 



Charles Aldrich: Developmental Assistant, Instructional and Training 
Systems Program (ITS Program), FWLERD. 

Charles Weynard Bailey: Educational Administrative Consultant, California 
State Department of Education, SEA representative on the Consortium 
Board of Directors. 

Bela H. Banathy: Director, ITS Program, FWLERD, FWLERD representative on 
tne Consortium Board of Directors. 

Herman D. Bates: Instructor rn Social Science and Psychology, Canada College, 
Instructor, Communications Skills course. Fall 1972, Canada College. 

Robert Bennett: Assistant to the Chancellor, San Mateo Community College 
District, Developer, Paraprofessional Program Implementation System, 
Canada College (of the San Mateo Community College District). 

John DeCecco: Professor, Psychology of Education^ California State University 
of San Francisco (CSUSF), Developer, Communications Skills course, 
EP level . 

James Dunn: Director, Developmental Systems Division, American Institutes 
for Research (AIR), Developer, Information/Data Collection course, 
various EP and PP modules, module 1, Evaluation course, and module 1, 
Developmental Engineering course, EP level, AIR representative on the 
Consortium Board of Directors. 

Paul Ehret: Superintendent of Schools, San Lorenzo Unified School District, 
Local Education Agency (LEA) representative on the Consortium Board of 
Directors. 

Freeman Elzey: Coordinator, Systems Development, ITS Program, FWLERD, 

Developer, various EP and PP modules in the Information/Data, Evaluation, ' 
and Planning and Design courses-. 

Jack Fraenkel: Professor of Department of Interdisciplinary Studies in 
Education, CSUSF, Instructor, Dissemination and Marketing course. 
Fall 1972. 

Darrah Hallowitz: Research Intern, ITS Program, FWLERD. 

George Hallowitz; Chairman, Department of Educational Administration, 
CSUSF, Developer, Entry Professional Implementation Program at CSUSF, 
CSUSF representative on the Consortium Board of Directors. 

John Helmick; Vice President and Director of the Western Office of ETS, 
ETS representative on the Consortium Board of Directors. 
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''^"^ ?ho''p..n?l-f " °^ ^LERD, Principal Investigator for 

^5%ho r.'°"^l-^°T*^!)^^ Training Program in DD&E, ex-officio member 
of the Consortium Board of Directors. "lenmer 

John Hourigan: Developmental Associate, ITS Program, FWLERD. 

^?3nr^J,-nn^'?JiKl°'r' department of Interdisciplinary Studies in 
hducation, CSUSF, Instructor, Evaluation course, CSUSF, Spring 1972 
Developer, module 4-, Evaluation course. ^ ' 

'^'""^oLSn^pn;./?'*'"'*""' Pl^""^'"9 Design course, 

Deveopmental Engineering course, Fall 1971, Spring 1972, Fall 197? 

Srs^o'n PMeve/'" ^^'""'"^ and Design and Communication Skills 

Bennett Kilpack: Instructor, Canada College, Comraunicationr Skills courso, 
Ueveloper, modules in Communications Skills course, PP level. 

Nancy Adelson McCutchari: Senior Developer, ITS Program, FWLERD. 

Robert McMenamin: Technicon Information Systems, Developer, Communication 
Skills course on PP and EP levels, Technicon Information Systems 
representative on the Consortium Board of Directors. 

Earl Miller: Professor, Department of Educational Administration, CSUSF, 
instructor. Developmental Engineering course, CSUSF, Fall 1971. 

Marcia Moore: Research Intern, ITS Program, FWLERD. 

''^"'aid ?e?sinn"el ?y1?ems,"?neve?"'' '''''''' '''''''''' Implementation 

Lionel Olsen: Professor, Department of Educational Administration, CSUSF, 
instructor. Planning and Design course, CSUS'F, Fall 1971, Fall 1972. 

^^""^ r";-'' -t^^^^^"^ Psychologist, Stanford Research Institute, 

fn Materials, Planning and Design course, Stanford 

Research Institute representative on the Consortium Board of Directors. 

Wayne Rosenoff: Coordinator, Materials Development, ITS Program, FWLERD. 

Hnoch Sawin: Professor, Department of Education, CSUSF, Developer, modules 
in Information/Data and Evaluation courses. Instructor, Information/ 
Data course. Fall 1971, Fall 1972. 

Monica Schmitz: Coordinator, Implementation System, ITS Program, FWLERD. 
Diana Studebaker: Research Intern, ITS Program, FWLERD. 

Elaine Taylor: Senior Staff Scientist, HumRRO, Developer, Evaluation 
Materials, Communication Skills course, module 2 for Developmental 
Engineering course, HumRRO representative on the Consortium Board 



James Upton: Instructor, Canada College, DeVetoper, modules for Information 
Data and Evaluation courses, PP level, instructor, Information/Data 
course. Fall 1971, Fall 1972, Evaluation course. Spring 1972. 

Joseph S. Ward: Coordinator, Assessments Development, ITS Program, FWLERD. 

-NoiTnan E. Wallen: Professor, Department of Interdisciplinary Studies in 
Education, CSUSF, Instructor, Planning and Design course. Fall 1972, 
Advisor, Engineered Internship, CSUSF, Fall 1972. 



B. Vitae of Far West Consortium Personnel 

21 

CHARLES L. ALDRICH 

DEVELOPMENTAL ASSISTANT, INSTRUCTIONAL & TRAINING SYSTEMS PROGRAM 
FWLERD 



Academic Background 

. B. A. San Jose State College, 1963 

. M. S. California State University, San Jose, Cybernetic Systems, 1972 
Professional Experience 

. Federal Grants Administrator, San Jose State College; Development work 
for University of California Urban Extension, Santa Cruz; Operations 
Officer, U. S. Aniiy Recruiting Station, Atlanta, Ga.; Vice-president 
(co-owner) Banner Play Bureau, Inc. . 

. 1971 -present. Far West Laboratory for Educational Research and Development. 
Assignments including development and field test coordination for the 
learning team form of the graduate-level, transportable training program. 
The Educational Information Consultant: Skills in Disseminating Educational 
Informatio n. 

Professional and Service Organizations 

. Member, advisory board. Community Response,, Inc., San Jose, Ca. ; member. 
Board of Directors, Santa Clara County Volunteer Action Center, San 
Jose, Ca.; Environmental Sciences Institute; Society for Cybernetic 
Systems. 

Publications 

. The Educatio nal Information Consultant: Skills in Disseminating Educational 
Information (with B. Banathv. W. Rosenbff. et al FWLERD. Berkeley, fa., 
1971; A Compendium of New Careers within SantaTlara County , (with L. 
Barozzi), U. C. Urban -Xtension, Santa Cruz (unpublished manuscript), 1970; 
"Feedback as a Function of Interpersonal Coriwnunication," paper given 
before the Society for Cybernetic Systems,/San Jose State College, 
San Jose, Ca., November, 1970; "Communication Technology," paper given 
before the Society for Cybernetic Syster^s, San Jose State College, San 
Jose, Ca., November, 1969. ' 
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CHARLES WEYNARD BAILEY 

EDUCATION ADMINISTRATIVE CONSULTANT 

CALIFORNIA STATE DEPARTMENT OF EDUCATION 



Academic Background 

. B. A. University of Redlands, 1936 
. M. A. University of Redlands, 1953 
. td.D. University of Southern California 1958 

Professional Experience 

. School Superintendent, Colton Joint Unified School District, 1966-69; 
. Assistant Superintendent and Superintendent, Colton High School Distric 
1 962-66 ; 

. Teacher, principal and Assistant Superintendent, Colton Elementary 

School District, 1937-59; 
. Instructor, University of Redlands. 

. Development of innovative project'* for ESEA Title III, dissemination 
of selected projects, coordination of Fiscal Management operations. 



Professional Organizations and Honors 

. American Association of School Administrators; 
. California Association of School Administrators; 
. Educare - University of Southern California; 

. Phi Delta Kappa; 
. Delta Epsilon. 
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BELA H. BANATHY 

DEPUTY DIRECTOR, DIVISION II 

DIRECTOR, INSTRUCTIONAL & TRAINING SYSTEMS PROGRAM ' 
FWLERD 

Academic Background 

, B. S. Hungarian Royal Academy, 1940 

. M. A. San Jose State College, Counseling and Psychology, 1963 
. Ed.D. University of California, Curriculum and Instruction, 1966 

Professional Experience 

. Deputy Director, Comnuni cation Program; Design and Test Curriculum Infor- 
mation Systems and Educational Planning and Management Systems; Principal 
Investigator, Design, Development and Validation of a Transportable 
Instructional System for the Training of Educational Diffusion Evaluation 
Personnel; Defense Language Institute; Development and testing of generic 
models for foreign language training systems; Designing generic models 
for aptitude and proficiency testing; Designing faculty training programs. 

. Designing, developing, and velidating systems for leadership training 
(1959-69) i Analysis of educational and training programs of school dis- 
tricts and other institutions; Consultant to schools, training institutions, 
and development agencies; Teaching professional courses and graduate se- 
minars in education, systems developir,ent and systems theory. 

Professional Organizations and Honors 

. AASA; AERA; MLA; Society for General Systems Research; National Task 
Force on Systems Education; Phi Delta Kappa, ASIS. 

Publications 

• Th e common concept foreign language test (CTB, 1962); A design for 
leadership development ( BSA, 1963); Instructional systems (Fearon Pub.. 
1 968 ) ; The design and management of training: A systems approach ( Boy 
Scouts World Bureau, 1969); Current trends in college .curriculum: A 
Systems approach (The Encyclopaedia Britannica, 1969); Systems and~ 
Education (San Jose State College, 1969); Systems development in guidance : 
A 1 earni nq -task-centered-approach (O.E. Bureau of Research, 1969); 
Several articles published in professional journals, (1960-1972). 
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HERMAN DEAN BATES 

DEPT. OF SOCIAL SCIENCE & PSYCHOLOGY 
CANADA COLLEGE, REDWOOD CITY, CA. 



Academic Background 

. A. B. Western Reserve University, 1953 
A. M. University of Michigan, 1962 
Ph.D. University of Michigan, in progress 



Professional Experience 

. School Psychologist, San Carlos Elementary School District, 1970-present; 

Pupil* Personnel Director, Enterprise Elementary School District, 
. Redding, Ca., 1969-70; Director and Chief Psychologist, Warner Guidance 
^ Center, Palm Springs, Ca. 1967-69; Staff Psychologist, Patton State 

Hospital, Patton, Ca., 1964-67; School Psychologist, Lincoln "^ark 

School, Lincoln Park, Mich., 1961-1963; School Psychologist, Dearborn 

City Schools, Dearborn, Mich., 1960-61. 

Professional Organizations and Honors 

. American Psychological Assn.; Western Psychological Assn; Calif. State 

Psychological Assn.; Calif. Assn. of School Psychologists and Psychometrists; 
School Psychologists Assn. of San Mateo Co.; National Council on Family 
Relations; Calif, State Marriage Counseling Assn.; and local mental 
health and community organizations, 

. Phi Delta Kappa 



Publ i cations 

. "Changing Attitudes Toward Mental Retardation Through Work Service 
Programs," MPA, 1960 
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ROBERT L. BEN^ETT 



Academic Background 

B. S. Montana State College, Physical Science, 1950 
M.S. Eastern Montana College, Guidance and Counseiinq 1959 

Kn^^:;^^t?o;^1^J^"^'^' ^^"-^^--^^ CurricJSanTlchod 



Professional Experience 



tducational Resources fpntpy- lo^c cy n r'^^* 'si-i^- m program, 

of San MatL Co^^^^^ PACE ce"^ Consultant, development 

state Colleges Member cai^^^^^^^^ Chancellor, Calif. 

Professional Organizations and Honors 

. California Junior College Association; CROOTS Committee 
Publications 

Its for the Future^ UA Rpnn^^t ^nv^ c^m^^^-.^^^i n^' ■ 



ize 

^bsociation of 
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JOHN D. DsCECCO 

PROFESSOR OF PSYCHOLOGY AND EDUCATION 
CALIFORNIA STATE UNIVERSITY AT SAN FRANCISCO 

Academic Background 

. B. S. Allegheny College 1946 

. M. A. University of Pennsylvania 1949 

. Ph.D. University of Pennsylvania 1953 

. .Post-doctoral study at Wayne State Un.iversity, 1953-55, 
Michigan State University, 1955-60, and University of 
Maine, 1956-57 (summers). 

Professional Experience 

. Erie Public High Schools, Erie, Pa. 1946-48. 

. University of Detroit, 1953-55. 

. Michigan State University, 1955-60. 

. University of British Columbia, Summer, 1963. 

. Columbia University, 1968-70. 

. New York University, Spring, 1970. 

. Unim Graduate School (for experimenting colleges and universities), 1971-. 

. California State Dept. of Education on Prograimed Instruction, 1961-63. 

. Hillsborough School "District, 1962-63. 

. San Francisco Unified School District, 1963-66. 

. Educational Testing Service Development of Field Test in Education, i 969-70. 

. Center for Urban Education, New York, 1970. 

. Editorial Consultant, CRM Books ( Psychology Today ), 1968- 

. Editorial Consultant to American Educational Research Journal , 1967-73. 

Professional Organizations and Honors 

. AERA: Divisions B, C, and D; APA: Divisions 2, 15, and 26; CERA; NCEM; 
AAUP; and Phi Delta Kappa. 

Publications 

. Nine books in the fields of Education and Psychology and numerous 
articles in professional journals. 
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JAMES A. DUNN 

DIRECTOR, DEVELOPMENTAL SYSTEMS DIVISIONS 
AMERICAN INSTITUTES FOR RESEARCH 
PALO ALTO, CALIFORNIA 

Academic Background 

• B. S. Wayne State University Mathematics 1954 

. .M. A. Wayne State University Educational Psychology 1959 

. Ph.D. University of Michigan Education and Psychology 1962 

Professional Experience 

Visiting Fellow, Laboratory for Human Development, Harvard University 

Assistant Professor Psychology, and of Education, University of Michigan 

Program Director, Project PLAN, American Institutes for Research (1968-70) 

0f'M?cMgan^'096l67r'''' ''''' ''''''''' '''''''''' 

Director, School Psychological Examiner Program, University of Michigan 

Professional Organizations and Honors 

. USOE Senior Post-Doctoral Fellow, Harvard University; USPHS Fellow, 
University of Michigan; Horace E. Rackham Fellow, University of Michigan. 

* S?f?,-!n^'^°''°^'"^ Association; American Educational Research 
Publications 

• A comparative study of p upil construct systems relevant to cla<;<;r nnm 
co^tjons^an^^venti (U. ofltiT h-igan, 1962h Dimensional TET-^FTF ^ 
test anxiety scale for children (Michigan Acad emy of Science, 19631 ; 
Training and certification of midwestern pupil personnel wn rkpr<; (u 

Of Michigan, 1967); The PL AN instructional prnnLm- . cvc;l^-/ "pp^,,pu 
o curriculum developip HFTA LRA, 19/0); Bias^ minimiza lilf n quest o^^'^^ 
(with collaborators, in preparation). ^ 



28 



PAUL EHRET 

SUPERINTENDENT OF SCHOOLS 

SAN LORENZO UNIFIED SCHOOL DlfJfRICT 

SAN LORENZO. CALIFORNIA 



Academic Background 

♦ A. B. University of California, Political Science & English 
M. A. University of California, Educational Administration 

> Graduate Study: University of California; University of Chicago; 
Teachers' College, Columbia University; and University of Virginia* 

Professional Experience 

♦ 1948-present, Superintendent of Schools, San Lorenzo Unified School 
District; 1946-48, Deputy County Superintendent of Schools, Alameda 
County; 1941--46, Officer, U.S. Navy, retired as Lt. Commander; 1938- 
1941, Teacher and Counselor, Berkeley, California Unified School Dist.; 
Consultant, Alameda Unified School District; Consultant, Davis 
Unified School District. 

Professional Organizations and Honors 

• . Past President: California Association of School Administrators; Oakland 
Area Council, Boy Scouts of America; Oakland Area Community Chest; Board 
of Trustees, Alameda Co. United Fund. 

. Past Chairmr^n: Financing Public Education State Committee, California 
Teachers' Association - 8 years; California School Administrators State 
Cooperative Finance Committee - 3 years; California Association of School 
Administrators Annual Conference, 1968; Region XII Delegation to Boy 
Scout World Jamboree, Asagiri, Japan, 1971. 

. Present Memberships: President, San Lorenzo Scholarship Foundation; 
President, San Francisco Bay Area Council, Boy Scouts of America; Member, 
American Association of School Administrators; Member, Association of 
California School Administrators 

. Past Memberships: Vice President, Bay Area United Fund; Member, Board of 
Directors, California Assn. of School Administrators; Member, Board of 
Governors, California Assn. of School Administrators; Member, Board of 
Directors, Alameda County Chapter, American Red Cross; Member, American 
Assn. of School administrators Study Mission to the Soviet Union, 1959; 
Leader, American Association of School Administrators Study Mission to 
the Soviet Union, 1968; Member, California Council on Public School 
Long Range Finance Planning; Member, State Council of Education, Califor- 
nia Teachers' Association. 
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FREEMAN F. EL2EY 

COORDINATOR, SYSTEMS DEVELOPMENT 
INSTRUCTIONAL &" TRAINING SYSTEMS PROGRAM 
FWLERD 

Academic Background 

. B. A. San Francisco State College Psychology 1957 
. M. S. San Francisco State College Psychology 1959 

Professional Expen'ance 

. Research Assistant in Psychiatric Research, Mount Zion Hospital, 
San Francisco, 1956-58. 

. Research Positions at San Francisco State 

. Research Associate in Mental Retardation, 1958-68; Research Associate 
in Project on Thinking in Elementary School Children, 1963-66; Co-director 
ilccci^^r *° "^^velop a Vocational Competence Scale for Mentally Retarded adults 
lacl cc ^°-9^^ector of Project to Develop a Pre-School Social Competence Scale, 
1965-66;. Senior Research Associate directing Field Observation Staff of 
bausalito Teacher Education Program, 1966-69; Research Director of Demon- 
stration Project for Nursery School Cross Cultural Education, 1966-69; 
1969 r Pi^e-School Project for Multiple Handicapped Children, 

. Lecturer in Education, San Francisco State College, 1967-71. 

. Consultant in Educational Research, Far West Laboratory for Educational 
Research and Development, Berkeley, California, 1971. 

Professional Organizations and Honors 

. American Psychological Association; American Association on Mental 
Deficiency; American Educational Research Association; California 
Educational Research Association. 

Publications 

♦ A^Proqrammed Introduction to Statist ics. 2nd Edition, Monterey, 8rooks-Cole 

♦ A First Reader in Statisti cs, teterey, Brooks-Cole, 1967 

♦ Business Statistics: A Programmed Introdi '-tion . Monterey, Brooks-Cole, 1971 

♦ A^Programmed Introduction to Research (with S. Levine), Belmont, Ca. Wadsworth, 

♦/ A Programmed Introduction to Educational and Psychological Measurement 
(with S. Levine), Belmont, Ca., Wadsworth, 19'70 ~ 
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JACK FRAENKEL 

PROFESSOR, DEPT* OF INTERDISCIPLINARY STUDIES IN EDUCATION 
SAN FRANCISCO STATE COLLEGE 

Academic Background 

. B.A* University of Nebraska at Omaha, Nebraska, Sociology 

M.A, San Francisco State College, San Francisco, Ca., Social Science 
Ph.D. Stanford University, Stanford, Ca., Social Studies Education 

Professional Experience 

• Jr. High and Senior High Teacher, Pacifica and San Francisco, Ca.;" 
Research Assistant, Secondary Education Project, Stanford University, 
1965-66; Assoc. Profeiisor of Interdisciplinary Studies in Education, 
San Francisco State College, 1966-71; Professor, Dept. of ISED, San 
Fr?.ncisco State College, 1971 -present. 

. Associate Director, Taba Curriculum Development Project in Social 
Studies, San Francisco State College, 1966-69; Coordinator, NDEA 
Institute on Teaching Disadvantaged Children, Sausalito Unified 
"School District, 1968; Assoc. Director, NCERD Project, Teacher Corps, 
(competency-based educcttion); Visiting Professor of Education, Dept. 
of Curriculum & Instruction, School of Education, University of 
Washington, Seattle, Summer, 1971. 

Professional Organizations & Honors 

. San Francisco Council for the Social Studies; California Council for 
Social Studies; National Council for the Social Studies; California 
Teachers' Association; American Association of College' and University 
Professors; AERA; Association for Supervision and Curriculum Develop- 
ment. 

. Pni Delta Kappa; Pi Gamma Mu; Alpha Kappa Delta. 
Publications 

. Helping Students to Think and Value; Strategies for Teaching Social 
Studies, Prentice-Hall, Inc., 1972; Teacher's Handbook for Elementary 
Social Studies, Revised ed., with others, Palo-Alto, Addi son-Wesley, 1971 
'Teaching about Dissent and the Draft" Intercom, New York, Center for 
War/Peace studies, Jan. -Feb., 1971; "Program Definitions: Logic and 
Process," The High School Journal , Chapel Hill, Univ. of North Carolina, 
April, 19701 Peacekeeping: Problems and Possibilities , with others, 
New York, World Law Fund, 1970; Crime and Criminals: What Can We Do 
A bout Them ? Prentice-Hall, Inc., 1970; and many more. 
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RESEARCH INTERN, INSTRUCTIONAL & TRAINING SYSTEMS PROGRAM 
FWLERD 

Academic Background • 
. A. A. College of Marin 1969 
. B. A. Sonoma State College 1971 

. M. A. California State University at San Francisco Sociology (in progress) 
Professional Experience 

. Employed with FWL s-,nce September 1971. Presently vorUng on development 
in Instructional Training Systems. ueveiopment 
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GEORGE HALLOWITZ 

CHAIRMAN, DEPARTMENT OF EDUCATIONAL ADMINISTRATION 
CALIFORNIA STATE .UNIVERSITY AT SAN FRANCISCO 



Academic Background 

• B. B. A, College of the City of New York. 1934 

. M. S. Cqlumbia University, Social Work, 1941 

. M, A. Southern Methodist University, Psychology, 1955 

. Ed.D. University of California at Berkeley, 1959 



Professional Experience 

. Director of camp, youth and community agencies; Training director. Peace 
Corps and VISTA ^raining at California State University at San Francisv^o ; 
Former chairman of Department of Interdisciplinary Studies in Education, 
California State University at San Francisco; Executive secretary of 
Study Committee on Curriculum Review at CSUSF. 

. Research and teaching in education; study of administrative behavior 
research training programs; direction, of community agencies. 



Publ ications 

. "An Observation InstruTOnt for Theory Oriented Research into Administrative 
Behavior." (Co-authors, Fibish and Hart), California Journal of Educa- 
tional Research 5 November, 1963, 



Professional Organizations and Honors 

. Association of State College Professors of Educational Administration; 
Vice-president, Academic Senate, CSUSF; President, CSUSF Chapter, 
American Association of University Professors, 



33 



JOHN S. HELMICK 

VICE PRESIDENT, DIRECTOR WESTERN OFFICE 
EDUCATIONAL TESTING SERVICE 
BERKELEY, CALIFORNIA 



Academic Background 

. B. S. Northwestern University, Psychology, 1940 
M. A. Wesleyan University, Psychology, 1942 
Ph.D. Stanford University, Psychology, 1942 



Professional Experience 

. Program Director for the development of the Admission Test Graduate 
Study in Business; Supervised development of materials for New York 
City first grade assessments, "Let's Look at' Children." 

. Instructor, Assistant Professor, University of California at Los 
Angeles, and University of Hawaii; at ETS since 1952, Vice President 
since 1963. From 1963-68 responsible for ETS instructional programs 
including workshops for foreign students, summer programs for graduate 
students in measurement, etc. 



Professional Organizations and Honors 

. American Psychological Association American Educational Research 
Ass(x:iation; National Council on Measurement in Educaiton; American 
Association for the Advancement of Science. 



Publications 

. l^roup factors in simple and discriminatory reaction time, "(with others); 
Studies in Motion Sickness ," (with others); "Attempted pupillary con- 
ditioning at four stimulus intervals," (with others); "Validity of test 
Items for measuring learning specific to a course;" "Tests can predict 
success; ' "Pursuit learning as affected by size of target and speed of 
rotation;' "A workbook for College Psychology ;" "Reliability or variability;" 
"Piaget for first grade teachers." ^ 



erJc 



34 

PAUL D. HOOD 

ASSOCIATE LABORATORY DIRECTOR FOR PROGRAMS, DIVISION II 

FAR WEST LABORATORY FOR EDUCATIONAL RESEARCH AND DEVELOPMENT 



Academic Background 

. B.A. Franklin College, Social Psychology, 1950 
M.A. Ohio State University, Social Psychology, 1950 
Ph.D. Ohio State University, Social Psychology, 1953 

Professional Experience 

. Director, Communication Program; Organizer, Bay Area T.V. Consortium, 
(Human Relations Training for School Staffs); Principal Investigator, 
Design of a Survey for Training and Personnel Requirements for Educational 
R2D.^D&E; Principal Investigator, Development and Testing of Procedures 
to Evaluate and Disseminate Information on Training of R&D Personnel; 
all at Far West Laboratory for Educational Research and Development, 
Berkeley, Ca. 

. Senior Staff Scientist, HumRRO; research" in training and utilization of 
low aptitude personnel; development, validation and implementation of 
Army-wide NCO Leadership Preparation Program; development and implementa- 
tion of Army Drill Sergeant Program; consultant on revision of Army 
Basic Training; 

. Director, Bomber Research Unit, U.S.A.F.; research on training, training 
simulators and evaluation of B-52 aircrews; 

. Research Associate, Personnel Research Board, Ohio State University; 
measurement of crew coordination; research on aircrew composition, leader- 
ship, and survival training. 

Professional Organizations and Honors 

. Fellow, American Psychological Assn.; Fellow, American Sociological Assn.; 
American Educational Research Assn.; American Society for Information 
Science; Western Psychological Assn. 

. Sigma Xi; Alpha Psi Delta (Grad. Psych. Honorary, Ohio State); University 
Scholar, Ohio State University, 1949-50. 

Publ i cations 

. Reports in HumRRO publication series on training research, leadership 
development programs; reports in Air Force Personnel and Training Research 
Center (USAF-AFPTRC) series on aircrew training, performance effectiveness 
and survival training; Far West Laboratory publications on educational 
information systems; training and arrangements for educational knowledge 
utilization. 
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JOHN HOURIGAN 

DEVELOPMENTAL ASSOCIATE, MATERIALS DEVELOPMENT 
IMSTRUCTIf^NAL ^. TPAININR SYSTEMS PROGRAM 
FWLERD 

Academic Background 

. B. S. Siena College, Loudonville, N. Y. , Mathematics, 1951 

M. S. University of Southern California, Instructional Technology, 1968 

Ph.D. University of Southern California, Instructional Technology and 
Educational Psychology, in progress. 



Professional Experience 



. Instructional materials development. Far West Laboratory for Educational 
Research and Development, 1972-present; 

. Instructional materials development, educational system analysis, mar- 
keting proposals and seminars, advanced instructional systems, computer 
assisted instruction, system project management. System Development 
Corporation, Santa Monica, Ca. 1956-72; 

. Industrial Engineering, George S. May Co., Chicago, 111., United Air 
Lines, Chicago, 111., and Department of the Army, Albany, N.Y. 



Professional Organizatictis 

.'American Educational Research Association 

. Association af Educational Communications and Technology 

. National Society of Programmed Instruction 
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HAL JONSSON 

PROFESSOR INTERDISCIPLINARY STUDIES IN EDUCATION 
CALIFORNIA STATE UNIVERSITY AT SAN FRANCISCO 



Academic Background 

. B. A. University of California, Berkeley 1950 

. M. A. University of California, Berkeley 1958 

. Ph.D. Ur.iversity of California, Berkeley 1964 



Professional Experience 
. Elementary and Jr. High School Teacher, West Oakland, 1954-57 

. Supervisor of Elementary Education and Assistant to Director of Demon- 
stration Schools, U.C. at Berkeley, 1957-58 

. College Teaching, San Francisco State College, 1958 to present, U.C. 
Berkeley, 1961-62. 

. Director of Teacher Corps projects from February 1969 to 1971. 

. Coordinator for Frederick Burk Elementary School-College observation- 
demonstration program, 1960-62. 

. Consultant Work: Evaulation Consultant, Offi \5 of Compensatory 
Education, California State Dept. >f Education, 1965-66; Evaluation 
Consultant, Teacher Corps, Washington Office, U.S.O.E., 1968; Evaluation 
Consultant, ESEA Title III Project, Pittsburg, California, 1967-68, and 
Title I Project, Berkeley, California, 1965-68; Research and Evaluation 
Consultant, STEP, 1966-68. 



Professional Organizations and Honors 

. AERA, NCME, CTA (CCUFA), ACSCP, AFT, Phi Beta Kappa; Conmittee 
Representative to Planning Committee, AERA panel, Sacramento, 1967. 



Publ i cations 

. "Interaction of Test Anxiety and Item Difficulty in Mathematics 
Problem Solving Performance," National Council on Measurement in 
Education Journal , (in press) 
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KENNETH D. KENNEDY 

DEPARTME^^• OF POLITICAL SCIENCE 

CANADA COLLEGE, REDWOOD CITY CA 



Academic Background 



A.A, College of San Mateo, History, 1962 
MA I'n State College, Political Science, 1965 

iinL '-^''2 Political Science 1966 

University of Kentucky, now being completed 



Ph.D. 

Professional Experience 



L°nI?e1f"?:^^dl\Sfrj:!l96T-6r.1l5f./3°"^^^^''^^^ Resident. FacuUy^ 
Professional Organizations and Honors 

■ ^oliaEio'n"'"'"' W«tern Political Science 



Pi Sigma Alpha 
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BENNET B. KILPACK 

CANADA COLLEGE 

REDWOOD CITY, CALIFORNIA 

Academic Background 
. B. S. Southern Oregon College Sociology 
. M. S. Southern Oregon College Sociology 

Professional Experience 

. Director, Child Development Center, Jackson County, Oregon 

. Supervisor of Staff Training, Orientation, Intensive Treatment and 

Out-Professing (Counseling Department), Thiokol Job Corps Center, Clear- 
field, Utah 

. Instructor, College of San Mateo, Sociology Department 

. Director-Administrator, University Day School, Menlo Park, California 

. Director, Juvenile Hall, Del Norte County Probation Department, 
Crescent City, California 

. Counselor, Ashland High School, Medford Senior High School 

. Assistant Director, Bar "0" Ranch for Boys, Del Norte County Probation 
Dupartment 

. Senior Group Supervisor, Hillcrest Juvenile Hall, Belmont, California 
. Assistant Director, Upward Bound, Pacific University, Forest Grove, Oregon 
. Assistant Professor, Sociology, Pacific University, Forest Grove, Oregon 
. Director, Head Start -Program, Jackson, Oregon 

Professional Organizations and Honors 
. State of California - Life Adult Credential 
. State of California - Life Junior College Credential 
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NANCY ADELSON McCUTCHAN 

SENIOR DF.VELOPER, INSTRUCTIONAL & TRAINING SYSTEMS PROGRAM 
FWLERD 



Academic Background 

• A, Mount Holyoke College, Political Science, 1965 

• A, California State University at San Francisco, Educational D,D&E 

(in progress) 



Professional Experience 

• Education Program Specialist, Title I, involved in identification, retrie- 
val, and preparation of information and evaluation reports on local 
projects (1965-67), 

. Far West Laboratory for Educational Research and Development 

1967-70, participated in design, development, and writing of packaged 
information units on elementary science and secondary social studies 
curricula, and in design and preliminary'development of the ALERT 
information system, 

1970-72, served as product development spe alist, course form field 
test coordinator, and editor for the graduate-level, transportable 
training program. The Educational Information Consultant: Skills in 
Disseminating Educational Information , 

1972-present, development spe alist for series /modules of D,D&E training 
program for entry and paraprofessional level personnel- 



Publications 

, The American Government Information Unit: Curriculum Alternatives for 
Secondary Schools (ED 052 114), with others, Berkeley, Far West Labora- 
tory for Educational Research and Development, 1970, 



Far West Laboratory program and evaluation reports (with others)^ 



ROBERT K. McMENAMIN 
INSTRUCTIONAL TECHNOLOGIST 
TECHNICON INFORMATION SYSTEMS 
MOUNTAIN VIEW, CALIFORNIA 



Academic background 

. B. S. Southeast Missouri State College, Education, 1954 

. Additional courses in Personnel Specialist, Tecljnical Instructor, 
Personnel Management and Data Systems, Instructional Programming, and 
Effective Writing at Air Force schools. 



Professional Experience 

. Instructional Programmer— conducted validation of instructional program 

for Personnel "Data Systems. 
. Prepared a task analysis, wrote objectives, performed. S-R inventory 

for teaching points, and wrote and edited. 
. Taught courses in Personnel Data Systems, prepared audiovisual aids, 

developed curriculum, and wrote and edited student workbooks and 

study guides (USAF). 
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C. EARL MILLER, JR. 

PROFESSOR OF EDUCATIONAL ADMINISTRATION 
CALIFORNIA STATE UNIVERSITY AT SAN FRANCISCO 

Academic Background 

. B. A. University of Idaho," 1948 

. M. A. Washington State University, 1954 

. Ed.D. University of California at Berkeley, 1960 

Professional Experience 

. High School teacher and administrator in Central Idaho, 1948-56. 

. Part time graduate student at UCB and High School teacher at Piedmont 
High School, California. Active in local CTA-affiliated teachers' 
organization, 1956-68. 

- Full time student and graduate assistant at UCB. Participated in a 
number of projects sponsored by the Field Service Center, 1958-60. 

. Employment in the Department of Educational Administration, California 
State University and part time consultant for the Solano County Super- 
intendent of Schools, Fairfield, California (7 years); Reed Union 
School District, Tiburon, California (3 years); The Coordinating 
Council for Higher Education (1/2 year) and the San Francisco Chamber of 
Commerce (several meetings), I960 to present. 

Publications 



. Numerous mimeographed reports while at UC. Extensive report writing 
at the Solano County Office of Education. 

. Golfo, Armand J. and Miller, Earl, Interpreting Education Res earch , 
William C. Brown, Publishers, 1965. ilecond edition entitled Inter- 
preting Educational Research, 1970. 
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MARCIA MOORE 

RESEARCH INTERN, INSTRUCTIONAL & TRAINING SYSTEMS PROGRA.M 
FWLERD 



Academic Background 

. B. A. Carleton College, 1965 

• Graduate seminar in Psychology, Harvard University, summer, 1966 
. M. A. San Francisco State College, D,D&E, in progress 1971-72 

Professional Experience 

. Substitute teacher, Boston Public Schools, Boston, Mass., 1966; 

. Research Assistant in Clinical Psychology, Harvard University, 1966; 

. Research Assistant, Curriculum Evaluation project. Upward Bound program, 
based in Boston, 1966-67; 

. Research Assistant, Office of the Mayor, City of Boston, and liaison be- 
tween the Mayor's Office of Public Services and the Public Schools, 
1968-69; 

. Editorial and research work on ALERT Project, Far West Laboratory for 

Educational Research and Development, 1970-71; 
. Research Intern, D,D&E Consortium, FWLERD, 1971-72. 
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PATRICIA PALLISTER O'BRIEN 
DEAN OF WOMEN 
CANADA COLLEGE 
REDWOOD CITY, CALIFORNIA 

Academic Background 

. B. A. Michigan State University Philosophy 1958 

. M. A. University of Maryland major in student personnel admin. 

minor in counseling and guidence 
internships in student activities 
and placement offices 

* ^n?rJn^^'ci!*f College student personnel administration 
sLmer of "oef ^"^^^""^^t^' summer 1966 and Stanford University, 

. Graduate work at University of Arizona, Stanford University and 
University of Santa Clara, 75 graduate semester hours beyond B.A. 

Professional Experience 

• Co~P^k!l?Xnd! '''' "^^^1^"^' 

. Student Assistant (Education), Bureau of Indian Affairs, U. S. Depart- 
ment of Interior, Washington D,C., prepared a report of the guidance 
program in Federal Indian Schools. yuiudn^e 

. Assistant Dean of Women, University of Arizona^ Tucson, Arizona. 

' Activities Directo?"'^"' California, Counseling and Student 

Dean of Women and Counselor, Canada College, Redwood City, California 

Professional Organizations and Honors 
. American Personnel and Guidance Association 

. American College Personnel Association 

National Advisory Commission TV - The College Student 1971-1974 

. California Teachers Association 
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1964 
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LIONEL R.- OLSEN 

PROFESSOR OF EDUCATIONAL ADMINISTRATION 
CALIFORNIA STATE UNIVERSITY AT SAN FRANCISCO 



Academic Background 

, B. A. University of the Pacific, 1946 
. M. A. Stanford University, 1947 
, Ed.D. Stanford University, 1956 

Professional Experience 

, Teacher and counselor in secondary schools; teacher in elementary 
schools; teacher and counselor in junior college; Dean of Student 
Personnel in junior college. 

. County schools of secondary education, child welfare and attendance, 
and Director of Guidance. 

. Assistant Superintendent of schools for curriculum development and 
special services in city school district. 

. College teaching: Long Beach State College; California State University 
at San Francisco; Stanford University. 

. Consultant to: 

Western Association of Schools and Colleges accreditation teams (high 
school accreditation); 

State Department of Education, Bureau of Compensatory Education; 

New Haven Unified and Desert Sands School Districts (in conducting surveys 
of the instructional programs); 

California Teachers' Association Professional Standards Commission. 



CARL H. RITTENHOUSE, SENIOR RESEARCH PSYCHOLOGIST 
EDUCATION RESEARCH PROGRAM 
URBAN AND SOCIAL SYSTEMS DIVISION 
STANFORD RESEARCH INSTITUTE 

Academic Background 

. B. A. (1946), M. A. (1948), and Ph.D. (1952) in psychology, 
Stanford University, also studied at Wilson Teachers College, 
George Washington University, and Temple University. 

Professional Experience 

. Head, Training Group, Philco Corporation, Palo Alto, California 
. Research scientist, U. S. Army Leadership Human Research Unit, 
Presidio of Monterey, California; conducted research in leadership, 
leadership training, and tactical and administrative decision-making. 
. Research psychologist, U. S. Air Force Armament Systems Laboratory; 

worked in areas of perception and motor c kills 
. Research assistant in psychology of music-, Stanford University 
. Project leader, educational information utilization studies 
. Project leader, technical manpower transferability study 

Project scientist, field experiments at the Research Office of 
the U. S. Army Combat Developments Command Experimentation Center, 
Fort Ord, California 
. Project scientist, U. S. Army Concept Team in Vietnam. 

Professional Organizations and Honors 

. American and Western Psychological Association; Phi Beta Kappa; 
Sigma Xi ; Listed in American Men of Science and Who's Who in the 
West; certified psychologist in the state of California. 



Publ i cations 
. Articles in various technical journals. 
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WAYNE ROSENOFF 

COORDINATOR, MATERIALS DEVELOPMENT 
INSTRUCTIONAL & TRAINING SYSTEMS PROGRAM 
FWLERD 

Specialized professional competence 

° Developing tests and instructional materials, programmed learning, 
dissemination and marketing of instructional products 

Representative R&D assignments 

° Project Director, EIC Training Project: The Educational Informat ion 
JLOQguItantL - uills in Disseminating Educational Inform ation, 

^ 1970-present, FWLERD 

° Director, Development Component, Instructional and Training 
Systems Program, June, 1972-present, FWLERD 

° Staff consultant for development of the Putting Research into 
■^^""tiO"a1 P'^actic_e_ (PREP) Information Unit, an activity of NCEC, 

^ USOE: Accountability and educational evaluation," FWLERD 
Director of Marketing, Director of Curricular Publications, 
Assistant Director of Test Development, California Test Bureau/ 
McGraw-Hill, 1955-70 

° Supervised activities of 16 field representatives throughout the 
U.S., conducted sales training conferences, planned product 
workshops on "A systems approach to individualizing instruction." 

Other professional experience 

° Conducted a 1972 AERA postsession 

Academic background 

• ° Ed.D., educational psychology, UCLA, 1957 
° M.S., physical education, UCLA, 1950 
° teaching fellow, graduate studies, UCLA, 1940-42 

teaching credential. University of Washington, 1940 
° B S., physical education, sociology. University of Washington, 1935-39 

Publications 

° Lessons fo r self-instruction , a programmed multi-level series in 

reading, math, and English, CTB/McGraw-Hi 11 , 1967 
° Strengtheni ng the student's learning "through in dependent study, 

^ CTB/McGraw-Hill, 1967 

° The educati onal information consultant ; skills in disseminating 

educational information . Berkeley, California: FWLERD, 1971 

Professional associations and honors 



o 
o 
o 



American Educational Research Association 
California Educational Research Association 
Adult Education Association 
° Phi Delta Kappa 

ERIC 
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ENOCH I. SAWIN 
PROFESSOR OF EDUCATION 

CALIFORNIA STATE UNIVERSITY, SAN FRANCISCO (CSUSF) 



Academic Background 

. B. S. University of Chicago, Mathematics, 1947 
M. A. University of Chicago, Education, 1948 
Ph.D. University of Chicago, Education," 1951 

. Post-doctoral; Stanford University, 1957-68 



Professional Experience 

. Faculty, Syracuse University, 1950-52; Professor (GS-13), Air Univer- 
sity, Maxwell Air Force Base, Alabama, 1952-60; Professor, California 
State University, San Francisco, 1960-present; Educational research, 
curriculum development and evaluation, developing new training programs, 
training others in educational research. 



Professional Organizations and Honors ^ 

. • American Educational Research Association; California Educational 
Research Association; American Psychological Association; Association 
of Supervisors of Curriculum Development. 



Publications 

. On writing team (Ch. 7 and 8) for 1958 A.S.C.D. Yearbook, A Look at 
Continuity in the School Program; "Broadening the Base in Evaluation," 
School Review. Spring 1959: "A Prnpnf;eH Model in Evaluation," (with 
Hilda Taba) In Educational Leadership. October, 1962; "The Air Force 
ROTC Curriculum Evaluation Project "(with J. F. Smith), in Improving 
College and University Teaching . Spring, 1966, pp. 81-86; member of 
committee that prepared Taxonomy of Educational Objectives. (D. R. 
Krathwohl , et. al_. ); Evaluation and the Work of the Teacher , 
Wadsworth, .1969. ' ' 
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MONICA SCHMITZ 

COORDINATOR, IMPLEMENTATION SYSTEMS 
INSTRUCTIONAL & TRAINING SYSTEMS PROGRAM 
FWLERD 



Academic Background 

♦ B. A* University of Minnesota, Sociology, 1967 

♦ M. S- (in progress) California State University at San Francisco, 

Industrial Psychol ogy 



Professional Experience 

. Group Living Supervisor, Hennepin County Juvenile Detention Center, 
1963-68. 

. Information and Referral Specialist, Pilot City Regional Center, 
(Minneapolis, Minn.), 1968-69. 

. Placement supervisor and office manager for a business-oviented 
personnel agency, 1969-70. 

. Instructional materials development, Far West Laboratory for Educational 
Research and Development, 1 970 to present. 



Publications 

• The educational information consultant: skills in disseminating edu- 
cational information , an instructional system, with Bel a Banathy, 
Wayne Rosenoff , et al . , Berkeley, Far West Laboratory, 1972. 
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DIANA P. STUDEBAKER 
RESEARCH INTERN, INSTRUCTIONAL & TRAINING SYSTEMS PROGRAM 
FWLERD 



Academic Background 

. B. A. Mills College, English & French, 1964 

. M. A. University of California, Berkeley, Comparative Literature, 1966 
. Secondary Credential, State of California, English & French, 1966 

Professional Experience 

. 196G, substitute teacher, Oakland public schools 

. 1568-71 Berkeley Y.M.C.A., clerical and publicity work. 

. 1971-present, Far West Laboratory, clerical, editorial and limited 
development and production responsibilities for D,D&E Consortium. 
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ELAINE N. TAYLOR 

SENIOR RESEARCH SCIENTIST 

HUMAN RESOURCES RESEARCH ORGANIZATION 

MONTEREY, CALIFORNIA 



Academic Background 

. B. S. Pennsylvania State University, Physical Education, 194S 
M. A. Bowling GreeR State University, Psychology, 1954 
Ph.D. State University of Iowa, Psychology, 1954 



Professional Experience 

. Senior Research Scientist, performance of low aptitude personnel, 
U.S. Army; Senior Research Scientist, preparation of programmed instruc- 
tional materials, NCO leadership course, U.S. Army; Director, Design, 
Analysis, and Editing, Fort Benning, Georgia; Member of faculty at 
Conference on "Collaborative Styles in Community Mental Health Services 
for Children and Youth," sponsored by the State of California Dept. of 
Mental Hygiene, June 1-2, 1972. 



Professional Organizations and Honors 

. American Psychological Association; American Association for the 
Advancement of Scitiuc-j. 

. The Society of Sigma Xi. 

. Winner, 1972 Publications Contest Awards, Society for Technical Communi- 
cation Publications Contest, Washington D.C. Area, HumRRO Technical 
^ Report 71-18 - Preliminary Hand book on Procedures for Evaluating Mental 

; Health Indirect Service Programs in Schools, (with Ernest K. Montague). 



Publications 

• Performance in Five Army Jobs by'Men at Different Aptitude (AFQT) Levels: 
I. Purpose and Design of Study , 2. Development and Description of fnstru 
ments, 3. The Relationship of~AFQT and Job Experience to Job P erformanceT 
4. Relationships between Terformance Criteria, (with Robert Vineberg), 
HumKKU, November 1970 and June 1972; Considerations on the Design of 
Instructional Systems for Employer Based Career Education, HumRRO, Hay, 
^9/z (With others}; Summary and Review of the VOLAR Experiment, 1971 ; 
Installation Reports for Forts Benmng. Bragg, Carson and Ord , and 
Permanent Party Studies^ " (with Robert Vineberg), HumRRO. April. 1972; 
Preliminary Handboo k"o n" Procedures for Evaluating Mental Healt h I ndirect 
iiervice P r ograms in"^oo ls, HumRRO Technical Report 71-18, August 1971 
(with Ernest Montague); Effects of A p titude (AFQT), Job Experience, and 
Litera cy on Job Performance; Summary of HumRRO Work Units UTILITY and ' 
KhALlSiiu , (With otTiets), HumRRU. -eb. 1971; and many othsrs. 
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JAMES MORGAN UPTON 

CANADA COLLEGE 

REDWOOD CITY, CALIFORNIA 
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Academic Background 

. A. B. Gonzaga University Honors Classical 1957 

. A. M. Gonzaga University Philosophy 1958 

. M. S. Seattle University Mathematics 1962 
National Science Foundation, Sunmer Institutes 

. Santa Clara University, Theological Studies. Three of 
four years toward S. T. M. degree. 

Professional Experience 

. Bellarmine High School, Tacoma, Washington, 1958-1965 

. Seattle University, Instructor in Philosophy (Logic, Metaphysics) during 
Spring Quarter, 1964 and Instructor in Honors Program, acadennc year of 
1 964-65 

. Instructor in Philosophy (Seminar on Teilhard de Chardin) during Sprinq 
Quarter, 1965 . / a h a 

. San Jose State 

. College of San Mateo 

Professional Organizations and Honors 
. Provisional General Certificate, Washington State, 1958-1963 
. Standard General Certificate, State of Washington, valid from July 1, 1961 
. Life Secondary June 1967, California 
. Life Junior College June 1967, California 
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JOSEPH WARD 

COORDINATOR, ASSESSMENT DEVELOPMENT 
INSTRUCTIONAL & TRAINING SYSTEMS PROGRAM 
FWLERD 



Academic Background 

♦ A» Tulane University, Psychology, 1949 
M. A. Tulane University, Psychology, 1958 
Ph.D. Tulane University, Psychology, 1962 

Professional Experience 

- Senior Staff Scientist, HumRRO; Project Director for development of 
programs for combat skills, medical skills, and psychomotor skills; 
job analysis of combat skills; development of management f training 
courses; consultant on evaluation of Army weapons systems, training 
techniques, and combat doctrine. 

• Faculty, Dept. of Psychology, Tulane University; Faculty, Dept. of 
Education, Auburn University; Faculty, Dept. of Psychology, Monterey 
Peninsula College; Faculty, Dept. of Educational Administration, 
San Francisco State College. 

Professional Organizations and Honors 

. American Psychological Association. 

. The Society of Sigma Xi. 
Publications 

."Motivational Properties of Frustration: II, Frustration Drive Stimu- 
lus and Frustration Reduction in Selective Learning," Journal of 
E xperimental Psychology , 1954, (with A, Amsel); Frustration and Persis- 
tence: Resistance to Discrimination Following Prior Experience with the 
Discriminanda , (with A. Amsel; Development and Evaluation of an Inte- 
grated Basic Combat/Advanced Individual Training Program for Medical 
Corgsmen (MOS 91A10), HumRRO, 1970 (with others) ; Evaluating Proficiency 
in the Use and Maintenance of Infantry Weapons, Eighth Annual Army 
Human Factors Engineering Conference, 19^, (with Thomas F. Nichols), 
Design of a Functional Competence Training Program for Development , 
Dissemination, and Evaluation Personnel at Professional and Paraprofes- 
slonal levels in Education, Dec. 1970. FWLERD (with P. Hood and others): 
Final Report: A Functional Competence Training Program for Development 
Dissemination and Evaluation Personnel at Professional and Paraprofes- 
sional levels in Education, Dec. 1971, FWLERD (with P. Hood and others): 
Development Strategies Used by the Far West Consortium for D,D&E 
Training," April, 1972, AERA Conference paper. 
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NORMAN E. WALLEN 

PROFESSOR, DEPARTMENT OF INTERDISCIPLINARY STUDIES IN EDUCATION 
SAN FRANCISCO STATE COLLEGE huulhixun 



Academic Background 

. A. B. University of Rochester, Economics, 1950 
Ed.M. University of Rochester, Educational Psychology, 1952 
Ph.D. Syracuse University, Psycliology, 1956 



Professional Experience 



, Research Instr., Dept. of Special Education, Syracuse University, 19 55 - 56', 
Statistical Consultant to Faculty and Students, Dept. of Psychology, 
Syracuse University, 1955-56; Ass 't. Professor, Dept. of Ed. Psych, Uni- 
versity of Utah, 1956-60; Assoc. Professor, 1960-65; Professor, 1965-67; 
Professor, Dept. of Interdisciplinary Studies in Education, San Francisco 
State College, 1967- present. 

. Director, three USOE Research Projects, 1961-66; Phi Delta Kappa Research 
Project, 1966; Taba Curriculum Development Project, 1967-69; Psychologist, 
Tn -County Mental Health Traveling Clinic, 1962-66; San Juan Mental Health 
loco^ln"^ Clinic, 1963-65; Consultant, Tri -University Project, Evaluation, 
1968-70; Institute for Staff Development, Miami, 1968-present; San 
Francisco Unified School Dist., 1969-70; School Desegregation Study, Uni- 
versity of Wisconsin, 1 969-present; Consulting Editor, Taba Social Science 
Program, Addi son-Wesley , 1971-present; Research Coordinator, STEP project, 



Professional Organizations and Honors 



. American Psychological Association; American Educational Research Associati 
American Association of University Professors; American Federation of 
Teachers. 

. Phi Delta Kappa; Sigma Xi . 

Publ i cations 



• Activity Bo ok for People in Communities (with M. Wall en), Addi son-Wesley, 
Menlo Park, 1972; Research and Teacher Education. Final Report (v.-th 
others). State of California Division of Compensatory Education, 1970; 
The Taba Cu rriculum Development Project in Social Studies. Final R eport , 
(with others), USOb Project No. OE-6-10-182, Addi son-Wesley , l969;~'1iTH~ 
proving Elementary Social Studies: An Idea Oriented Approach", Elementary 

School Journal , 70, 154-163, 1969, (with others); "The Use of Teaching 

Modules to Study High Level Thinking on the Social Studies," Journal of 
Teacher Education. 18, 495-502, 1967, (with others); et al. 



Appendix E: TRAINEE INFORMATION 



Tables and Resumes 

Entry Professional Program: 

California State University at San Francisco, 

Cycle I and Cycle II students 

Paraprofessional Program: 
Canada College, 
Merritt College, 
Contra Costa College 
students 



ENTRY PROFESSIONAL STUDENTS 

California State University at San Francisco - Cycle I students 
Nancy 

D.i,-4.-^"?^c^"^"*'^ Holyoke College where she received a B.A. deqree in 

sSl af? t anH"h^^ h'^' "^it'V": ^^^^^ ^" Education Progr S 

f ha f iears 5? thf MI'*^ ""'^ Laboratory for the past three and 

with thP HpupinL^J t a P^og^am Assistant, Nancy has been involved 

l-sLmina?ton T .-f -nl^^'^^^ f ^•'^^■"^■"S in information 

cnssem nation. She is interested in expanding this training through the D D&E 
program and in learning more about the development of systems 

Carol 



of rMr.°In*'^'?hrJ ^'h.^^- ^'3'"' "^'^^^^y °f f^o"! the University 

of Chicago. She taught for a year at Walden School in Berkeley and has done 

In lll^f' ^^^^"^ (Califomia) public lihoo?" Carol 

Reseat h ntern' l.'!nr'°'^^° ''''' T ^"^Pl^^^^ there as a 

Kesearch intern Her work involved research and writing about educational 

developments. Carol is interested in working as a curriculum consultant to 
schools and envisions starting a school some day. 

Marqot 

Margot received her B.A. in French and Art from Stanford University She 
has been extensively trained in metal s.ni thing, dance, and teaching and has 
worked professionally in all of those fields Margot's metllSork and lewelrv 

Joun rl dTv Scho T.rnlf\^''t''°' °l'^t Kindergarten Program cf ?he Ma?i-n 
country Day School. Margot has been with the Far West Laboratory since 1970 as 
a Program Assistant. At the Lab she has been the Director of the iZlJgl 

t 0 E'?h" rd'Se °l r^ll^^-TJ^o-^g^ ^as developed materials for Kindergarten 
tnrough Third Grade, has done classroom demonstrations of video-tapes and has 

us no'l. n'Srr ^'^'''T'f' V^^"^-"^^ ' ^"^ M^^g°t Is Interested n 
He .\nnu' \ho'.''"'"^V'''^°P "integrated" curriculum for use in pub- 
lic schools She hopes to become a consultant and possibly a teacher of 
various age levels, including adults as well as children.. 

Nancy 

Nancy attended Occidental College where she received her B.A. deqree in 
w?J; tS^''-H'"/^^l\'K-- ^^"^^'^^ twentieth century Bolivian nSvel 

Ass?.t.nt InH I '/"^'^'I^^^'^"^!!* *° ""^ P^^' She was a Research 

ToJ?^^Lo'^\^^^ Educational Research Council of America in Cleve- 
llrJ^ *M-^^- ^" ^^^^ Nancy worked in Menlo Park, California with 

Educational Consulting Associates as a Consultant. She later did consultina 

Sest 1^;; .'°n'^p1a7n'"'''^'S" '''''''' Sumy,.^e. Nancy has been wUh the Far 
th't r^nJ cJ ? Progrm Assistant in the Communication Program. In 
TnflfcE^ ^ ^^J designed and developed IPMS training units. Nancy is 
IKe D D&E Dro"graT ^"^ ^"^^^''^^^ °^ educational development work through 
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Table 2 
Entry Professional Student Data 
California State University at San Francisco 



Cycle I 
Fall '71 Sd. '72 Fall 


i Cycle II 
1 Fall 72 


Race: Caucasian 


i 

15 12 8 


1 

8 


Black 


9 ■ 


9 
0 


6 


8 


Oriental 


0 


0 




Sex: Male 


1 


2 1 


1 3 I 


Fema 1 e 


23 


19 13 


ill ' 


Department: 
ISED 


13 


12 7 


7 


Ld. Admin. 


10 ' 8 6 


10 


Other 




u — 


Internship: 

Teaching 


1 1 1 ; 1 


8 


Special Hub. School 


3 i 3 ! 3 ■ 


! 2 




20 . 17 i 10 


i 5 


Other 


i i 


. 

2 


Previous D,D&E experience: 
none 


6 


1 

; 4 


17 


0 - 2 yrs . 


10 ■ 


8 1 8 


0 


3 yrs. or more 


8 


6 ' 2 
1 


0 " 


D,D&E Course Enrollment: 
Planning & Design 


24 


X 


i 

3 


1 7 
1 / 


Info/Data 


24 


X 


0 


17 


Communication Ski 1 Is 


X 


16 


X 


y 

A 


Evaluation 


X 


15 


X 




Dev. Engineering 


X 


14 


X 




Uisseml nation/Marketing 


X 


X 


12 


' 0 


tngineered Internship 


24 


10 


0 


17 


Non-D,D&E Courses Taken: 
one course 


10 


5 


5 


0 


two courses 


5 


7 


1 


0 — 


three courses 


0 


1 


3 


0 


more than three 


0 ; 

i 


0 


1 


0 


Age: 20 - 25 yrs. ' 8 i 


9 


I 

2 ' 


3 ' 


2b - 30 yrs. - 10 i 


7 


7 


r "1 


3£r& above "! 5 ' ' ' 


4 


4 


5 



Q * information not available at tSi t ; li? % ,n,f ifff^rpd 
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Celia 

Celia received her B.A. from the University of California at Berkelev in 
Socio ogy. She minored in English and Spanish. Celia hafbeen with the Far 
West Laboratory for the past eight months as a clerk-typi t She is interested 
in expanding her knowledge and bettering her career opportunities in various 
areas of educational research through her involvement in the D,D&E program. 

Doris 



Colleae'at utllllf °J Science degree from California State 

Ca ?orni. K J ^"^ her e ementary teaching credential from the University of 
Ul forma. She taught fifth and sixth grades for one year at Lonofellow Srhnn 

continuing her wo?k in the puE^if chools and w L S^^ 
d'd&E prSg^am. "^''^ °^ leadership through her involvement -in Se 



Patricia 



\n.Jr IV-^ received her A.B. degree in English from the University of 
wn^Sn fn^'"'- *'"9ht grades three, five, and six for four years. Sh'^ 
wa. a t.?^n'?nrc^5'" ?W "'^^ Education Division of Xerox where she 
was a training specialist. Patricia has been with the Far West Laboratory for the 
•^nH oi";-^"'"! ' '^■^Sram Assistant, a position which entails research? writing 
1- is^•n'th T^J^nT'"''^- is interested in expanding her knowTe^^^^^ 



Marie 



Marie received her B.A. in Speech and Drama from the College of 
bt. Catherine. She worked for five months with Materials for Today's Learnina 

yearsl^S ''Xi>ha.'r'M-'^"f J ^"^lish. and Forensic^ for h e' 

years at St. Michael High School. She has been involved in directing a 



Lorraine 



D ^-.^'"S''''^^"^ received her B.A. in economics from Southern University in 
Baton Rouge. She substitute taught at the elementary and' secondary levels for 
IZ h^^..' LorraiL has been doin se etarial 

rZLZ ? i^f * and-a-half years at the Educational Testing Service in 
berKeiey. California. She wishes to pursue a career in Development. 
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Me redith 

Meredith attended the University of Arizona and received a B.A. from there 
in 1964 in Elementary Education. She taught second and fourth grades for five 
years in Orange County (California) and has been at the Far West Laboratory for 
one year as Research Intern in the department of Teacher Education. Meredith 
would like to develop competencies in D,D&E to aid her in her work at the Lab 
and to help prepare her for future work in the public schools. 



Li 1 1 i an 

Lillian received her B.A. in Social Work from the University of California. 
She worked for the Children's Home Society in Oakland for four and-a-half years 
before coming to the Far West Laboratory. At the Lab Lillian first worked 
for the Personnel Department and then became Administrative Assistant to the 
Utilization Division. She hopes to utilize her D,D&E skills in agencies like 
the Far West Laboratory. 



Barbara 

Barbara attended the University of Oregon where she received her B.A. degree 
in Journalism. She has been with the American Institutes for Research for the 
past five years researching and developing training materials. Barbara expects 
to improve her job efficiency through development of D,D&E skills. 



Darrah 

Darrah attended Sonoma (California) State College and received her B.A. 
from them in Sociology. She has worked for two summers as a counselor at a 
children's camp in San Rai -^1. Darrah is presently working with the Consortium 
at the Far West Laboratory as a Research Intern. She is interested in teaching 
sociology at the college level and developing materials for sociological 
studies . 



Carolyn 

Carolyn received her B.A. in anthropology from Stanford University. She 
worked as a Research As*^-*^^ *it at Stanford Research Institute for five months 
and then joined the Amer iv,.. ..istitutes for Research, also as a Research 
Assistant. She has been with them for the past eight months. Carolyn is 
interested in using her D,D&E skills to do work in educational research and 
development. 

Cynthia 

Cynthia received her B.A. in Psychology from the College of the Holy Names 
and her Elementary and Secondary Teaching Credential from the University of 
California. She worked'^for seven years at the Berkeley Recreation Department 
as Playground Leader and for one year as a sixth grade teacher at the 
Longfellow Elementary School. She is interested in studyincj Educational 
Administration through the D,D&E program. 
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Marcia 

Marcia attended Carleton College where she received a B.A. degree in 
Sociology. Since then she has earned some additional graduate credits in 
Psychology at Harvard University. Marcia worked for one yar as a research 
secretary for the city of Boston, for six months as a Research Assistant in 
the Upward Bound program at Harvard, and as a substitute teacher for one yar in 
the Boston public schools. She has been at the Far West Laboratory for almost 
two years as proofreader and copyeditor. Marcia is interested in organization 
and curriculum as it is applied in the development of alternative schools. 



Sheila 

Sheila received her B.A. in Early Childhood Education from the University 
of North Carolina. She taught for a total of seven years in North Carolina and 
in Richmond, California before coming to the Far West Laboratory. At the Lab 
Sheila has been a Research Intern for the past year-and-a-half and has been 
primarily involved with curriculum analysis. She is interested in pursuing a 
career in the public schools in a leadership position and hopes that the D,D&E 
program will aid her in this area. 



Ti miza 

Timiza received her AA in Social Science from Merritt College and her B.S. 
from California State College at Hayward. This past summer she completed the 
requirements for her Elementary Teaching Credential at the University of 
California at Berkeley. Timiza taught second and third grades for one year and 
preschool through Project Headstart for another year. She is interested in 
starting a private school for non-shite children and hopes that her studies in 
D,D&E will aid her in this endeavor. 



L.E, 

L.E. received a B.A. degree in Psychology from Sonoma State College, He 
worked at Oonoma State for two-and-a-half years as Audio-Visual Supervisor. 
L.E. has been with the Far West Laboratory for the past year and-a-half as 
Research Intern in the Multi-Ethnic Program. He is interested in developing 
D,D&E skills to help him devise relevant materials for use with non-white 
children in community schools. 



Kashan 

Kashan received her B.A. in Social Science and her Elementary Teaching 
Credential from San Francisco Rtate College in 1971. She has been with the 
Far West Laboratory for the past few months as Research Intern doing evaluation 
of Lab products. Kashan is interested in going into the development area of 
Education. 
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Cycle II Student Biographies 



Francis 

Frank received his B.A. in History from the University of California at 
Davis, continuing his education in the five year Intern Teaching program where 
he received his Standard Elementary Teaching Credential. He tuaght for five 
years in the Davis area, acting as Social Studies Coordinator (K-12) for two 
years. Frank has offered district and college inservice courses, was a member 
of the KQED Social Studies comnittee, has been active in the East Bay Council 
for Social Studies, and gave a Social Studies Workshop for the State Social 
Studies Convention held in Fresno. He is currently seeking a Supervisor of 
Curriculum Credential at CSUSF, and hopes that the D,D&E program will prepare 
him to develop curriculum programs. 

Lorna 

Lorna attended New York State University at Buffalo, where she received 
a B. S. in Mental Retardation and Elementary Education. She taught Special 
Education in the Buffalo School District for a year, and has taught in the 
Berkeley Unified School District for the past two years. She will be teaching 
5th graders this fall. Lorna is presently completing work for a California 
Life Credential. She is interested in entering the development area of 
education to increase her qualifications for working with non-white children. 

(Edie)- Marie 

Marie received a B.A. degree in English from CSUSF. For six years she 
worked for the U. S. Forest Service doing computer programming and documentation. 
She has recently become Public Service Careers Coordinator for Region V of 
the United States Forest Service. She plans to develop an employment program 
to hire, train, and counsel the disadvantaged. Marie intends to extend her 
activities into the community and public school systems, and she hopes that 
D,D&E training will help prepare her to develop these programs. 

El izabeth 

Elizabeth earned a B. A. degree from the University of California at 
Berkeley, taught at Oakland High School for three years, then moved to the 
Monterey peninsula where she taught at Fort and Monterey Peninsula College 
in the Evening Division. She is presently writing curriculum materials on 
journalistic skill development and teaching in Bennc-r Jr. High School in 
Sunnyvale. Elizabeth is interested in developing programmed and packaged 
materials with an affective and cognitive design for the junior high school 
level . 

Carolyn 

Carolyn received her B. A. in Chemistry from California State College in 
Hayward. She taught high school drop-outs in a Neighborhood Youth Corps, in 
Vallejo, California, for one year. During the past year, Carolyn has coor- 
dinated a Model Cities Federally Funded Educational program administered by 
OPS. She is interested in developing materials that are environmentally 
related. 
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Esther 

Esther attended the University of California at Davis, where she earned 
a B. S. degree in Child Development. She has worked with counselling, testing, 
and statistics, and spent a year as a program assistant with the Wright 
Institute administering. a Field Study Program. Esther hopes to enter research 
and curriculum development for handicapped children. 

Barbara 

Barbara has a B. A. in Art History, and taught grades 3-6 in the American 
School in Conakiy, Republic of Guinea, West Africa. She has an elementary 
teaching credental from San Jose State College, and taught 4th and 6th grades 
for two years and a summer school African enrichment program in the Ravens- 
wood School District. During the past three summers she has taught pre- 
and school age children in the San Francisco School District Childcare Centers. 

Marilyn 

Marilyn received her B. A. degree with a Social Science Field Major with 
an emphasis in Sociology and Psychology from the University of California' 
Berkeley. She is interested in designing and evaluating a curriculum to tfach 
Spanish to pre-school children through games, music; play and the arts. 

Jimmie 

Jimmie earned a B. S. in Elementary Education with a minor in the Social 
Sciences from the University of Nebraska at Omaha, then tdught 5th grade in 
the Omaha Public Schools for two years. After moving to Berkeley. Jimmie 
taught 4th grade for two years and has been a skills special i.st for a year, 
a job which entails working with both teachers and children in improving 
reading and math skills. 

Dianne 



_ Dianne received her B. A. degr3e in Classics from the University of 
Arizona. She is now employed in th-s Evaluation Division of the Early Child- 
hood Education Program at the Far West Laboratory. She is particularly in- 
terested in developing classroom materials which win be relevant, and 
disseminating them into the educational system. 

Carrie 

Carrie received her B. S. in Education from the University of Nebraska 
at Omaha. Following graduation she migrated to Oakland, where she has- been 
enployed by the Oakland Public Schools for the past six years. Since the 
school in which she works has a primarily Black student body, she is very 
interested in developing materials which will help Black children learn. 
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Major 

Major earned his B- S. in Sociology and Family Relations from Weber 
State College in Ogden, Utah. During his stay in Ogden he taught in the 
ethnic studies department and also worked there as a program and curriculum 
developer* He is now working as assistant coordinator of a federally funded 
Model Cities educational program administered by OPS* 

Rosemary 

Rosemary received her B* A* in Psychology from the University of Califor- 
nia at Riverside, She is currently a student at CSUSF and is particularly 
interested in educational research and the development of instructional ma- 
terials* She .will begin her internship at SRI this fall* 

Gail 

Gail earned a B* Ed* in English and Social Sciences from the University 
of Hawii* She has taught English in Hawaii, substituted for the Oakland 
Public Schools, and substituted and taught English to Asian-Americans in 
San Francisco* Her main interest is curriculum development slanted towards 
helping Asian-American students* 



i 
i 
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Paraprofessional Student Data 
Pre-Service Continuing Education 



Race : Caucasian 
Black 



Oriental 



Sex; 



Ma^e^ 
Female 



Internship : 
publ ic__s_chpol 

Paid 

'ToTunteer 



Junior College units 



already taken : 
0 - 15 
ir_- 30 

3Tr - "45 ■ 

45 - 60 & above 



Canada College Merritt College 

K'7I Sp.'72 F'72 F F ' 7rTSpT^72"^ 



31 
0 



1 



2 



27 
3 



25 
Id" 



14 
6 



11 
T9" 



72 



„ i-J 4 
' 0 "o 



4 

0" 



g 

"5' 



2 
'2 

T 



3 
5 



2 
2 

... 

5 
3 



1 'I 
•7- 



16 


6 


11 


13 


■■■ 4 


• -JT 


4 


4 


5 


3 


6 


• 11 



D>D&E course 
enrollment : 
Planning & Design 
FnTo/Data ^ 

.Communi cation Skills 
Evaluation 
Developmental 
^Engineering 
internship ^ 



6 


X 


23 


X 


■ "15 


X 


X 


15 


X 


6 


8 


8 



Non-D,D&E courses 
taken: 

pne_cp_urse _ 
two _courses" 
thre"e courses 
more" fharfthree 



3 
.1'7 
13 

X 
X 

3 



0 
6 

6 
0 



-J. 
5 



5 
0 



0 
5 

"0~ 

5 
0 



Contra Costa College 
F'71 Sp.'72!F'72 : 



J. 
0 



0 

A. 
0 



0 
_8 
0 



0 
8 



8 



0 
~8 



8 
0 



_8 

d" 



8 

"O" 



2_ ! 3 

2 ■" 2 

1 ■ 0 

1 ' 0 



3 

0 
0 



3 
3 



4 
4 



* 



Age: 



under 20 
20 - 25 
25.- 30 
over 30 



J 2 

1 


: 2~- 
-i 


1 


1 

1 9 


J 1,- 


1 

: 2 


; 21 




; 16 


r '4' 


i 8.„. 


. "4" 


1 2 


5 


8 



information not available at this t 



X 

4 
2 

X 



5 

X 

4 

X 



6 1 '5' 



J 
2 
0_ 
0 



2 
_0 



I 



0 

J_ 
1. 
4 



I 0 



5 

■4' 

X 

x_ 
"5 



± 
2 
0 



0 

1' 

X 
1 



4_ 
1 

T 
2 



i 1 



4 

T 



8 
'x 

'8 
'x" 



X 



_x_ 
'x 



J_ 

8 



X 

*8" 



0 
7" 



i 1 



0 
6_ 

2 



0 
6' 



* 



0_ 
6 

±1 



0 

"6' 
0 



iine 



X (tot offereu 



PARAPRnFRSSinNAL ^^TUPENTS 

I 



Canada College^ San Mateo Conimunity College District 

Gabriel - 

Male, married. Lives in East Palo Alto. Graduate of Abeokuta 
High School, Abeokuta, Nigeria. Completed 31 to 45 units of credit at 
Skyline College before enrolling in DD&E courses. 

Howard 

Male, single. Lives in San Carlos. Graduate of San Carlos High 
School, Completed 31 to 45 units of credit at Canada College before 
enrolling in DD&E courses. 

Lawrence 

Male, single. Lives in Redwood City. Graduate of Sequoia High 
School. First-time freshman student at Canada College. 

Mario 

Male, married. Lives in Redwood City* Graduate of Jefferson 
High School in 1947. Moved to this area from Seattle, Washington. Has 
completed 46 to 60 units of credit at San Francisco State College and 
other colleges before enrolling at Canada. 

William 

Male, single. Lives in Redwood City. Graduate of San Carlos 
High School. Has completed 16-30 units of credit at College of San 
Mateo. 

Barnett 

Male, single. Lives in Redwood City. Graduate of Woodside High 
School^. Born in Long Beach, California. Has completed 31 to 45 units 
at Canada College. 

Bruce 

Male, single. Lives in Menlo Park. Graduate of Woodside High 
School. Bora in Los Angeles, California. Has completed 15 units at 
Caliada College. 
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Gayle 



<:rhnn fll V "^^^^^^^^ .L^v^s in Redwood City. Graduate of Pioneer High 
In J 1?°''" completed 16 to 30 units at 

San Jose State College, and Palomar College, before attending Cafiada 



Lori 



uinh J?"^!^' T"^^ Redwood City. Graduate of San Carlos 

High School. Born in San Francisco, California. First-time freshman. 

John 

Male, single. Lives in Menlo Park. Graduate of Woodside High ' 
School. Born in Rochester, New York. First-time freshman. 

George 

H,-oh c^^^^' ^^n^^^'. ^^"^^ Me"^° P^"^"^' Graduate of Menlo Atherton 
High School. Born in Seoul, Korea. First-time freshman. 

Susan 

Female, single. Lives in Millbr;»e. Graduate of Capuchino High 
School. Has completed 16 to 30 units at College of San Mateo. 

David 

c . n^^l^'T^J"^^?- Lives in Menlo Park. Graduate of Minerva High 
Scnool, in Illinois. Has completed 15 units at Canada. 

Jimmy 

u- u c■^u^^* ^^""91^' Lives in Redwood City. Graduate of San Carlos 
High School. Has completed 15 units at Canada College. 

John L 

Malf^, single, ^ives in Atherton. Graduate of Menlo-Atherton High 
School. Has completed 15 units at Canada. 

•Sylvia 

u-«i, /f"^!^' Lives in Half Moon Bay. Graduate of Woodside 

High School Has attended University of Pacific, Stockton; and Cabrillo 

Sfi-l^f 'o ""-^ ^'""f' completed sixteen to 30 units before attending 
Cafiada. Born in Boston, Massachusetts. 

Shamin 

Male, single. Lives in Redwood City. Graduate of St. Joseph's 
r^ll^lt■ Cacutta, India. Born ^n Calcutta, India, is a foreign student 
completing 16 to 30 units at Canada College. 
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Linda 

Female, married. Lives in Redwood City. Graduate of Arroyo High 
School, San Mateo, California. Prev^"ously attended California State 
College, at Hayward, and Chabot College, completing up to fifteen units 
Born in Alameda, California. 

Timothy 

Male, single. Lives in San Carlos. Gru^uate of San Carlos High 
School. Born in Redding, California. Is complt'ing 15 units at 
Canada College. 

Richard ■ 

Male, single. Lives in Redwood City. Graduate of Menlo Atherton 
High School. Born in San Francisco, California. First-time freshman. 

Edmund 

Male, single. Lives in East Palo Alto. Born in Lagos, Nigeria. 
Is attending Canada on foreign student visa. 16 to 30 units. 

Samuel 

Male, single. Lives in Redwood City. Born in Tel-Aviv, Israel. 
Graduate from "Kalai" Givatain, Israel. Now taking 15 units at Canada 
College. 

John 



Male, single. Lives in San Mateo. Graduate of Hillside High School. 
Is completing 15 units at Canada College. Born in Evanston, Illinois. 

Thomas ' / 


Male, married. Born in Richmond, Virginia. Attended High School 
there. Completing 15 to 30 units. 

Don 

Male, single. Born in Pannipitiya, Ceylon. Attended high school 
there. Also attended Institute of Practical Technology, in Ceylon. 
Born in Ceylon, is a citizen of that country and attending Canada on a 
student v.isa, having completed up to 30 units. 

Arthur 



Male, married. Graduated from Escondido High School. Born in 
Calgary, Alberta, Canada, is an inmi grant. Previously attended Idaho 
State College in Rocatello, Idaho, Boise Junior College, in Boise, 
Idaho, and Merritt College in Oakland; having completed up to 60 units. 
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Elizabeth 

Female, married. Lives in Menlo Park. Born in Sharon, Connecticut. 
Graduated from Menlo Atherton High School. First-time freshman. 

Rosemarie 

Female, single. Lives in Redwood City, California. Graduate 
from Sequoia High School. Is completing up to 30 units at Canada. 

Frank 

Male, single. Lives in Menlo Park. Did not graduate from high 
school. Born in San Francisco. Is completing up to 15 units at 
Canada College. 

Barbara 

Female, single. Lives in Hillsborough, California. Graduate of 
San Mateo High School. Born in San Mateo. Is taking up to 15 units at 
Canada College. 

William S 

Male, single. Lives in Menlo Park, California. Born in Rochester, 

New York. Completed high school there. Also attended S.U.N.Y. in 

New York, and has up to sixty units. 

Nelia 

Female, single. Lives in Belmont, California. Born in San 
Francisco. Attended Notre Dame High School. 

David H 

Male, single. Lived in Hongkong, and is a citizen of Britain. 
Has attended Royden House College, and New Method College, apparently 
both in Hongkong. 

Donald 

Male, married. Lives in Menlo Park, California. Born in Oakland, 
California. Graduated from Westmoor High School. First-time freshman. 

Margaret 

Female, married. Graduated from Westmoor. Born in San Francisco. 
Is taking up to 15 units at Canada College. 
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Merritt Colleae 

Catherine 

Cathv -attended high school in Cleveland and later received a 
certificate of Secretarial Arts from the Ashville Industrial Education 
Center in North Carolina. Since then she has earned 70 credits in 
Afro -American Studies from Merritt College in Otikland. Cathy has worked 
as a legal secretary and as a medical secretary and has been at the 
Far West Laboratory for the past two years, first as a secretary and 
then as Office Manager. She hopes to gain more knowledge and profi- 
ciency in Educatiop,al Research and Davelopment and hopes to 30 on for 
a higher degree after she completes the DD&E program. 

Fannie 

Fannie went to high school in Memphis and attended Tennessee state 
College for one year in Business Education. She has taken courses since 
then at Merritt and at the University of California Extension. Fannie 
worked for two years at the post office as Mail Clerk and doing keypunch. 
She worked for the IRS from 1963 to 1965 retrieving and filing income 
tax returns. From 1965 to 1968 she did coding, supervising, and key- 
punch for the Bay Area Transportation Study Commission. Fannie has been 
at the Far West Lab since 1969 as Research Intern, a job which entails 
collecting and organizing data and field testing. "Her studies in DD&E 
pertain directly to the field in which sfie is working and she hopes to 
better her chances for advancement.. 

Nathaniel 



Nathaniel went to high school in Houston and then worked for five 
years as a merchant seaman. He has been at the Far West Lab for the past 
four months as Mail, Clerk. Nathaniel is interested in accumulating col- 
lege credits and would like to get some training so he can enter the 
field of Educational Research and Development. 

Olga 

Olga has done course work at Merritt College in the field of 
Liberal Arts. She has worked as a secretary since 1942 at Superior Sheet 
Steel (1942-47), Colorado State University (1953-56), United States 
Department of Agriculture (1962-70) and the Far West Laboratory (1970 to 

*5^nKpc^^"*^* ""^'"^^ *° '"^^'^^ her professional level' as a result 

Of DD&E courses and is interested in expanding her areas of knowledge and 
keeping up with change in the field of education. 
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Jacqueline 
Jean 

Hon..oo^^"cl!^'-''^" ^^'^''^9 '^^^^^S^ ""^ses and is working - towards an A A 
sL'7s"in resLSirSa fnT^'hro IIecT''''? ^"^ .the^pScl^De aft^'nt. 

Bobbie 

Clerk!°a„"'trre'e yel?r llVJi-r f ""f '"^"'^ ^^"'^^ " ^ Procurement 
credit. '^r^. ^ne is taking Che D,D&E courses to gain college 
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Contra Costa College 

Robert 

Robert attended Contra Costa Junior College for two years. He worked 
for a year at the South Side Center in Richmond as a typist and later with 
the Concentrated Employmant Program as Supply Clerk. Robert is now employ- 
ed at the Far West Laboratory as Research Trainee. He is interested in 
making social change through education. 

Deborah 



Deborah attended Contra Costa Junior College for six months. She 
worked there during the summers of 1969 and 1970. Deborah has also been 
employed by Montecino Elementary School in Martinez, California. She has 
been with the Concentrated Employment Program and is now employed at the 
Far West Laboratory as Research Trainee. 

Dexter 

Dexter attended Contra Costa uunior College for one semester in 
Business. He has worked for Safeway, Transbay, Standard Oil of Richmond, 
and Western Can Company. Through his interest in science and his involve- 
ment with the Concentrated Employment Program, Dexter obtained employment 
at the Far West Laboratory as a Research Trainee in the New Careers 
Program. 

Macky 

Macky attended the Mt. Hood Community College in Gresham, Oregon for 
the better part of a year where he studied music. He has worked with the 
White Stag Manufacturing Company, the Rheem Manufacturing Company, and 
Safeway s Richmond warehouse. Through his involvement with the Concentrat- 
ed Employment Program, Macky obtained employment as Research Trainee in the 
Far West Laboratory's New Careers Program. 

Georgia 

Georgia worked for seven years as a clerk typist and secretary, and 
two as a sales clerk before joining the Concentrated Employment Program. 
She IS now employed as a Research Trainee in the Career Education Program 
at the Far West Lab. ^ 

Linda 

Linda attended the Linton Business School for one semester and Contra 
oosta Junior College for one year. She has worked as a salesperson 
with C-Shore Sales and as a Clerk-Carrier at the Berkeley, California Post 
Office. She has been involved in the Concentrated Employment Program since 
April of this year and is now employed in the New Careers Program at Far 
West Laboratory as a Research Trainee. 



Ernest 



Ernest received a Certificate of Complttion from Prentiss Junior 
College in Prentiss, Mississippi in 1966. He attended Huston-Til lotson 
College in Austin, Texas the following year. Ernest studied Psychology 
and Child Development. He substitute taught at the junior high level 
in Gulfport, Mississippi and worked as a day care instructor in Austin, 
Texas. Ernest wishes to expand his career in education through his in- 
volvement in the D,D&E program and his employment at the Far West 
Laboratory as Research Trainee. 

Rosella 

Resell a attended high school in Fresno and Berkeley, California 
and majored in business. She worked as a waitress from 1966 to 1969. 
Rosella is now employed at the Far West Laboratory as a result of her 
involvement with the Concentrated Employment Program. 
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Executive Summary 

This executive surm«ry highlights and places a reconxnended priority 
on selected recommendations from the more lengthy report. For ease of 
locating the detail which buttresses the recommendation each sun«,ary recommend- 
ation Is followed by the page numbers In the report where the more detailed 
statement Is carried. 

1. Since the training package will be heavily dependent on the r.uallty 
of the Instructloniil materials. Instructional materials packages an. of the 
highest priority of any of the specific activities of the project during the 
pilot operational stage. The Instructional and curriculum design of these 
materials need to be evaluated and shaped in two Iterations by the end of 
1972. (PP. 13-16, appendix 3) 

2. An Internal evaluator should be appointed to the project. His main 
duties would be to organize and conduct the formative evaluation, coordinate 
the evaluation system activities, and super vise the preparation o{= the 
evaluation package for the operational training package. (PP. 12-14. 16-25) 

3. In order to Implement formative evaluat (on. transact lonal evaluation 
should be undertaken in the pilot operational phase of the program. This 
transactional evaluation conducted by either the internal evaluator or 

an external consultant, would Involve s;,udents. college staff, Intern 
superlvors and project staff. (PP. 9-13) 

4. Educational Testing Tervice should be asked to complete the forms 
and the manual on their recommended use. During the pilot operational 
phase these forms should be field tested and modified by ETS on the 

basis of these data. These will constitute a major component of the evaluation 
package. (PP. 26-27) 



The evaluation system needs to Improve Its efficiency in two ways: 
(1) to the present project shape In formative evaluation and (2) In the 
contribution of tools to the final training package. Conceptually these 
two objectives should be separated in organizing the present evaluatloi 
activities by the internal evaluator. (PP. 2-9) 
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The following description of servi cesfrequested in the contract ,yFWL 
72-025^. has served as the organizers of this consultative effort. 

1. Review the Consortium evaluation system to determine the degree 
to which it is meeting i ts operational goals. This review will include 
the full range of the Consortium's evaluation activities to include the 
information and data requirements of the various systems, data and 
information gathering i ni;truments , storage and retrieval and information 
flow. 

2. Make recommendations to revise and improve the Consortium's 
evaluation system and instruments. These reconwendat ions will be made 
to improve the operational effectiveness of the system in providing 
the information required to determine the degree to which the program 
is meeting its stated goals. 

More specifically this report focuses on the immediate needs of the 
evaluation system in an effort to make a usable input to the development 
and operational ization of the training system. The shortcomings of the 
evaluation system and its failure to contribute to the developmental 
process of the project are well documented in previous outside evaluations 
(see Eash, Clark, Seiber, Rabinowitz, and Hopkins). 

However, what is largely lacking in these reports are specific plans 
for implementing an effective evaluation and developmental system in the 
D, D and E program to assist the first stage operational goals of the 
training and development system, ^'Developing and Testing a Model for the 
Training of D, D and E Personnel." Our concern is the establishment of 
a viable evaluation system which will: (1) provide effective formative 
evaluation in the pilot operational stages of the program; (2) render 
evaluation data that will aid both decision making by personnel at various 
levels (students, instructor, supervisor, course developer, and project 
administrator and consortium administrators), and also provide specific 
correctives to the products of these efforts; (3) eventually emerge as 
a transportable evaluation system, as readily implemented as the other 
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parts of the operational training program. The bulk of this report Is 

directed to specific suggestions for immediate evaluation design and 
practice for the year 1972. 

Our concern for the immediate is heightened by the continued 
documentation through our own findings of the insufficiency of evaluative 
data in guiding the processjn^ producing a functioning training system. 
As will be explicated in the following sections, we believe the present 
state prevails because of conceptual misunderstanding of the staging of 
evaluation in the developmental and pi lot operational phase of the process 
and the resulting incomplete interface of evaluation with the process as a 
result of its placement with a subcontractor prematurely. 
I . is the Consortium evaluation system meeting its operational goals ? 

The operational goals of the evaluation systems as distinct from the 
training and development system are to use process and product evaluation 
procedures to provide data on both the developmental training program 
and the operational training program. (See the Evaluation System 
Section N) In an illuminating paragraph the authors of the original 
proposal describe the importance of distinguishing between the evaluation 
procedures for a developmental training program and the more finished 
training package, the operational training program. It is instructive to 
rs-read this paragraph for we believe it represents a significant distinc- 
tion in the written proposal that has become lost in the present plans 
and operation. 

Distinctions must also be made between the developmental training program 
and the operational training program. During program development ft will be 
necessary to take more different kinds of measures and more frequent measures 
than will be required in the operational phase, since the developing program 
will be subject to alteration arising from assessments made during the develop- 
ment process. The operational program may have to be altered as well but it 
IS assumed that most corrections will have been made by the time the system is 
operational, and that simpler and less frequent p^^rformance measures will, 
therefore, suffice, to insure functioning to decign capabilities and standards. 
Measures to be retained and the frequency with which they will be administered 
will be determined in the develop .enta! phase. auminiscerta 
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Because of the loss of this distinction in the actual evaluation design, 
the developmental training system during the pilot operational phase has been 
handicapped with lack of meaningful evaluation data and fails to respond to 
Immediate developmental problems. Moreover, the evaluation design provided 
by ETS Is applicable to the operational training program which does not 
exist. Confusion among the different role participants appears to exist in 
their expectancies for evaluation. The evaluation design for the program 
produces little useful data (useful to the course developers and project 
managers on current problems). The apparent lack of grasp of the essential 
Importance of providing an evaluation design related to the appropriate 
staging of the project is impeding current work. In short, the evaluation 
design provided by ETS Is for an operational training program not for a 
development in a pilot operational phase program from which the operational 
training program wl 1 1 evolve. This lack of use of differential evaluation 
data can make during the beginning stages of the program. The conceptual 
failure has accompanying antecedent problems in the misplaced responsibility 
for evaluation data and its implementation. Under the present operation the 
evaluation system cannot meet its present operational goals until recogni- 
tion is given to the need for differential evaluation and responsibility for 
its implementation specifically assigned and assumed. The following dis- 
cussion spells these problems out in more detail. 

^- Needed Di fferential Evaluation In the Present Training and Development 
System . 

The present evaluation design is handicapped conceptually by its failure 
to provide d-i f ferent la 1 evaluation during the development of the training 
system. As a result of this missing formulation, considerable confusion 
abounds both in the design developed by ETS and in the competing and contra- 
dictory requirements for evaluation held by different role participants. 
Using an evaluation design that ii largely appropriate for an operationaJ 
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training program, the ETS design fails to provide evaluation data geared 
to the present stage of development* As the situation now prevails, the 
evaluation design and participant expectancies place the pilot operational 
program under strain as it calls for evaluation data and judgments that 
the project cannot meet, or if it meets, distracts from instead of enhances 
the development processes. To illustrate the relationship of an appropriate 
differential evaluation design for the D , D and E project, a brief summary 
of the program stages in a differential evaluation are specified in Figure 
These stages demand different evaluation strategies and designs. 



FIGURE I 

Characteristics of Program Stage 
Differential Evaluation Design 
Initiatory Program Model 



Development Program Model 
(Pi lot Operational) 



1 . A priori mode I . 



2. Described but not 
tested. 



3. Incomplete in some 

parts and descriptions, 
Relies on logical rela- 
tionships. 



1. Field testing under way 
with subjects. 

2. Descrepancies between 
what will and v; i 1 1 not 
work on the £ priori 
model are restated. 

3* More complete, but not 
a total refinement - 
unpredictable in some 
of the results. 



Integrated (Operational) 
Program Model 

K A completed package, 
ful ly operational . 

2. Coherent parts, in- 
tegrated relationship 
among parts, clearly 
def i nes roles. 

3. If properly implemented 
evaluation data is 
gathered to correct 
operational stages as 
they relate to program. 

4. Pred I ctab le. 



The D, D and E project has an extensive a prior? program, but the 
project is only now moving into a pilot operational training program where 
the a priori nx)del will be translated into an operational program which will 



A fuller account of these is contained in the paper appended, Eash, Maurice J., 
'Mssues in Evaluation and Accountability in Special Programs For Gifted and 
Talented Children,'' June 1971 1 (Prepared for the USOE). 
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emerge with the qualities or a transportable training system. As an a prior? 
program the correspondence with reality is not a one-to-one equivalence and 
current evaluation theory calling for a direct comparison as the judgmental 
test between what has emerged against what was written constitutes a test of 
a Platonic ideal that can only induce stress in participants and warp in the 
training model. The first obligatior of a developmental program is to reality 
test the constructs of the initiatory, a priori program, and the assessment 
to be rendered is one of pragmetio practical ism not of a correspondence to 
an ideal state. As preliminary data indicates, there is need to modify some 
of the original outlines of the modules and the engineered internship. These 
difficulties in programming were not forseen in the original a prior i model. 
In the current limited evaluation data in hand this reality testing process 
now seems slowly underway. 

Moreover, there will be a considerable shifting of the initiatory model 
as the practical possibilities in the field testing become more clear and 
validity criteria for the program emerge. Tests- of the program as an inte- 
grated functioning training system are to be devised only after clearly 
formulated, stable model components have been developed. In short, during 
the pilot operational stage, evaluation should be concentrated on formative 
evaluation, formative in the sense of defining, refining, clarifying and 
stabilizing the model. Unexpected and unanticipated consequences are uncovered 
as a by-prod ct during this stage, and provision for coping with them in the 
integrated model are devised. 

In the present evaluation design and its conceptualization, the confusion 
between what is evaluation for the developmental model and what is appropriate 
for an integrated model is marked and interferes with evaluation effort. One 
outside evaluator's criticism of the current evaluation as being only a 
"curriculum materials evaluation design" belittles the project-'s need at this 



juncture in the developmental „«del stage (See Clark). Indeed materials 
are critical and the training systems transportability will largely hinge 
on the self instructional and independent implementation qualities of the 
finished product. The model cannot be an integrated training system until 
these modules are developed and the internship planned. Other demands for 
evaluation of the personnel and integrative systems, are probably premature 
at this time. Only as the training system begins to shake down and the inter- 
faces of the systems become clarified will the other systems be ready for 
n.ore the refined evaluation called for in an operational training system. 

To illustrate, the modules for training are now undergoing considerable 
revision with the first group of students and instructors. Given the prior- 
ities, the development of materials takes precedent over the evaluation of 
the personnel system. While some preliminary data on the personnel system 
can be gathered during this materials development cycle it will also be 
biased and influenced by the materials evaluation. Provision is made in the 
development cycle for three iterations of materials, it is possible that 
a full.r evaluation of the personnel system particularly on students must- 
await the second or third iteration of the materials. More probably the 
personnel system will be most rigorously evaluated when the operational train- 
ing program is functioning as an integrated training system. This will 
probably occur in 1973. 

As a first priority then, there is need to focus on formative evaluation 
for the pilot operational phase of the program at this time. Given the de- 
mands in developing the modules and internship, the other systems cannot be 
properly evaluated using the present group of enrolled students. The present 
evaluation design does not provide the needed data for the developmental 
training program to meet its goals of: (1) developing and testing a model 
for the traininc of D, D and E personnel; (2) further specifying the goals 
and objectives of the training programs; (3) designing, developing and 
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validating 0, D and E training modules; and (k) aiding training institu- 
tions in using the training modules and delivery systems in the formulation 
of relevant criteria; While there is soirre provision for the gathering of 
data which will be useful in the formative process, the vehicles necessary 
for bringing it into the development and design work are largely absent. 
In other sections of this report, suggestions are made for evaluation 
requirements and timing on the systems and for a vehicle to bring formative 
evaluation data into an interactive interface with the continuing effort on 
design and development of the tr. lining system. 

The first concern of the evaluation system must be to address itself 
to becoming a formative agent in the development cycle. At the present it 
is not. Moreover, unless this difficulry is attended to immediately, there 
will be even greater problems as more modules are developed. We believe 
the initl'il descriptions of evaluation in the original proposal are compre- 
hensive in their approach to evaluation and offer fruitful suggestions 
(see Section N) but the implementation of the evaluation strategies remains 
unresolved . 

' ' • What are the Information and Data Re'.u? rements of the Training Development 
System? 

Since the training development system is at a stage of translating an 
a priori program into a developmental program there is need for specific 
information on how the original (initiatory) program squares with the constraints 
and realities of the contexts, personnel, students and materials that constitutes 
a functional program. |n this developmental stage there is a need for the 
following specific categories of information: 

'1) What is the Quality and adequacy of the training modules which have been 
developed? 
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(2) How do these modules fit Into the total program design directed toward 
the fulfillment of the objectives of the D, D, and E Program? 

(3) What are the problems in the internship, both in locatino quality exper- 
iences, programming students, and relating supervisory and teaching 
personnel to the internship? 

Within the categories, a number of more specific questions are being spawned, 
but the evaluation system as presently constructed does not aid very much in 
promoting data gathering or in organization of data into a useable form. 
Furthermore the attempt to evaluate the pilot operational training program as a 
operational (integrated) program is creating confusion among the several 
role participants. Recognition of the di''ferences in these two stages of 
program development, through use of differential evaluation, should help 
clear up the present confusion. Bi}^ equally important is the need to use 
evaluation strategies which will brine evaluation data and the development 
process into a mutually supportive relati\(5nsliip. 

In the present stage of the training sys\'-em a considerable range 

of formative evaluation data is being gathered; The ETS design lists 

\ 

seven instruments that furnish data $ome of which \^n be cycled as feedback 
to program developers, instructors and part ici pants. \^i le there seems to be 
sufficient data for feedback and the preliminary interview data from students 
and instructors identified problems both in the materials and in the class- 
room instruction, in the present evaluation design there is no ^jdequate vehicle 
for bringing these data to bear as effective feedback. This, we beijeve, can be 
corrected by instituting a process for 'gathering and using data that does not 
separate the responsibility for acting on the data from the process of gathering 
data and malcing recommendations for action. Lfeing a procedure called transactional 
evaluation, the hiatus between gathering data, and acting on the implications 
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of findings can be bridged. If transactional evaluation is implemented, the 
role of the evaluator is broadened considerably from the one in present use 
and the process of design-developer and evaluator become much more closely 
allied. This is explicated further in the description of how transactional 
evaluation would ^unc-^on in the improvement of the modules, instruction 
and internship. 

A. Transactional Evaluation Design 

The purpose of transactional evalu-ition is to provide a vehicle and an 
environment for gathering and organizing evaluation data that provides for 
implementation and action on the findings. In some respects it parallels 
organizational development work, which is needed if the D, D, S E program is 
to function in a consortium. The following is a scenario of how transactional 
evaluation can meet the present project's current evaluation demands. 

The evaluator would meet with the students and staff responsible for 
the course nxidules at the two colleges. Since the programs represent different 
populations, the meetings should be held separately. (Also the meetings for 
the internships and supervisors might be held separate from the course instructors, 
though in some oases these may be the same individuals.) 

The purpose of this meeting is to generate data that clarify the 
participants' goals, the modules' goals, and strategies for the attainment of 
both. Following an introduction to the purpose of the meeting, the evaluator 
should ask the participants to respond in writing, with answers limited to a 
sentence or two, to the general questions: 

1) What are the problems in this course that are keeping it from being 
as effective as I think it should be?" 

2) What are the major problems in the classroom activities that are keeping 
them from being effective?" 



91 

3) "What are the m^jor problems in the materials that we have been using?" 
k) "Given that this program is different from vost college programs, 
what does this change mean to me as an individual?" 
(Due to previous information gained from the interviews that Sam Levine is 
processing, another question or two may be more suitable than those above.) 
The different participants .should code their papers S for student, C for college 
teacher, SI for supervisor of interns, etc. Once these comments are made the 
interviewers will need sufficient staff to prepare on the spot a questionnaire 
which uses the comments as statements with a strongly agree, agree, disgree, 
strongly disagree reponse format. Example:The materials in the modules we have 
had so far are far too easy: Strongly agree, agree, disagree, strongly disagree. 
In the preparation, the evaluator should attempt to use at a minimum one comment 
from each of the respondents, leaving it, if possible, in the original text to 
the extent that it can be identified by its author. These responses are 
collected by the evaluator and an instrument is built. We have found that 
dividing an evaluation meeting with a long lunch hour usually gives the ev:Juator 
sufficient time to prepare the instrument given enough secretarial assistance 
vo type on ditto a lengthy questionnaire, run off, and collate it. The evaluator 
* should make clear to the participants that he" wi 1 1 be including a few of his 

own items. From the data on the program's Tunctioning, we would suggest the 
following items be included. 

0 I feel that Mm not fully qualified to accept the role required of me 

in this new program, 
o Participants in this program don't feel that they will obtain a job 

after their graduation, 
o I don't know what I will be able to do if this whole program starts 
turning sour. 

ERIC 
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These items are Intended to defuse inter-institutional conflict, and to allow 
sentiments to surface that data now testifies exist. 

In the second session (P.Mj the participants are asked to respond to 
the statements and a compiling of responses is done immediately within the 
group. From these responses the evaluator proceeds to lead a discussion of: 

(1) those where there is largely agreement and elicits reasons for these 
responses, 

(2) those responses where there is considerable variation and records 
the reasons for the difference in responses. 

Using these data, small group task forces are formed to develop suggestions 
for revision of course or role performance, and a report is rendered to the group 
and discussed. Jn these reports suggested solutions, next steps in a plan of 
action should be included. In some cases 5t is helpful to have those who are 
responsible for the program to respond on constraints and problems of implemen- 
tation. It is advisable to keep the task forces to six or less in order to 
encourage focusing on several specific, but not overlapping problems. In some 
settings, it is also advisable to assign the same problem to two groups to work 
out suggested solutions. It is essential to record questions, plans of action, and 
solution for future use in development and evaluation. If future transactional 
evaluation sessions are held,, these data can be used to chart progress and 
provide check points. They are also check points for the evaluator and admin- 
istrators of the project. We anticipate that there may be a question on why 
Sam Levine's interview data could not be used t.o build a questionnaire serving 
the same purposes. These data lack the essential factor of gaining commitment 
from individuals through using the group setting to build the instrument. Using 
transactional evaluation as a vehicle in formative evaluation has the 
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advantage of lodging responsibility for acting on the evaluative data with 
those who are active participants - hence data does shape and form the " 
project^s activities as the definition of formative evaluation implies. 

One question for the D, D, & E project staff is who shall .ct as 
evaluator? Should ft be an outside agent (ETS. Sam Levine. or some other 
person) or can the internal evaluator role U handled by Joe Ward, and Freeman 
Elzey or an internal evaluator to be appointed? it may pose problems if the 
D, D, & E staff who have been closely associated with the materials development, 
or with supervision of the internships and are not seen as objective observers 
of the program assume the evaluator's role. We do not have enough data on 
the prevailing roles and relationships to make a more specific reconriendat i on at 
this time. However, we are persuaded that the evaluation design is weak on 
implementation of formative data and provision for use of vehicles such as 
transactional evaluation necessary. At this time we are of a mind that the 
responsibility for evaluation in the developmental model stage can not be 
lodaed m an outside aaen^ if effective formative feedback 

is to be obtained. The evaluation must be much more closely allied with the pilot 
operational work. Moreover, the evaluation must receive more consistent attention 
and be performed in concert with the development work. The project staff does 
not seem able to give it this attention, nor does the outside evaluator 
have the capability. Though Sam Levine's work is moving toward getting formative 
data which can be used as feedback, this appears to be more by accident than 
design, and there is no provision for him to function as a transactional 
evaluator. We therefore recommend that provision be made for an internal eval- 
uator within the project, especially during the pilot operational stage. 

Similar use of transactional evaluation can be made in the other program 
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activities of counselling and internships -fol lowing the general procedures 
outlined above. Since the production, refinement, specification, and use of 
materials is the most stressing problem, the illustration was drawn on this area. 
B. Fo rmati ve Evaluation o£ Modules 

Formative evaluation of the modules is an area of most immediate, obvious, 
and pressing concern, and three tasks should be accomplished accord-ng to 
schedule: a) a thorough review and critique of the substantive content of 
each module by an inhouse expert or outside expert in the area of the module; 
b) a detailed analysis of the instructional methodology within each modnle 
with careful attention to the match of stated goals and learning experiences 
provided, the continuity, sequence, and integration of the principles covered, 
the authenticity of the examples, etc.; and c) a c.^reful examination of the 
^^"l"^"^^ interfacing of content and method across the several modules. 
Each of these tasks is discussed in subsequent sections. 

a. Because many aspects of 0, 0, & E are controversial, a review of 
substantive content is necessary. This is not to say that Borg's "The 
Three Tests in the Minicourse Development Cycle", "The Task Concept Outline", 
and the more recent "View of Stages of Voduct Life" and "Evaluation Stages 
for Major Laboratory Products" would not br, useful guides to further develop- 
ment and formative evaluation of the modules. However, these proposals for 
evaluation assume that there is a unified, agreed-upon set of specifications 
on what constitutes valid content, for example, "The Specification of Expected 
Outcomes" , "The Consideration of Alternatives." We suggest that at least 
two experts who have not previously worked on the Project be commissioned 
to critically review the content of each module, it would be preferable to 
contract with two people of fairly different theoretical persuasions, for 
example, James Popham of UCLA and Elliot Eisner of Stanford on objectives, 
b. Another task to be accomplished in the formative evaluation of 
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the modules is an an;ilysis of the instructional methodology. We are pro- 
viding a self-explanatory form for this purpose in Appendix B. 

c. On several occasions during our interviews with staff members, 
they noted problems of continuity from one module to the next. It would 
seem worthwhile at this point to consider the balance of the self-contained 
quality of the modules against the elements of (1) content, (2) method, and 
(3) examples that would tend to lock the modules together. A curriculum 
matrix of the common elements in each module should be constructed* 

It is not altogether clear which agency or person should perform 
each of the three tasks. Outsiders, of course, might have more objectivity 
in criticising the modules; but FWL staff who have not worked on the D, D, 
and E Project might do almost as well in this respect. (L. Jenks) Perhaps 
a mix of insiders and outsiders within the scope of the projected budget 
might be best. The data which will be generated in the transactional evaluation 
sessions can also be brought to bear on the materials revision. 

Given the dependence of the program on these modules the development 
revision, and field testing of them should be given highest priority. Again 
responsibility needs to be formally and specifically lodged with an internal 
evaluator who will further specify and map out these evaluation strategies 
and prepare a schedule for their completion. The development of these modules 
to a state that they can be moved into an operational training package should 
be completed by the end of 1972. 

C» Evaluation Information and Trainees in the Developmental Trninin^ System 

Th'e trainees in the program at both S.F. and Canada College are in a 
program that is still undergoing development. Due to the present stage of 
development a test of the materials effectiveness as finished products cannot* be 
made. By the same token it is difficult to see how the materials would be 
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developed without a student body and an instructional setting for field 
tc-ting. The present on-going program has been further critic! ed on the 
grounds that the S*F. student body is atypical in being more experiences 
and better educated than one might expect to recruit in the program. Though 
the latter criticism may or may not be accurate, the pre^^ent student body 
still can be used to develop the materials end ready che operational 
training system. If this route is taken and we strongly recommend that ;t 
be followed, then the chief evaluation requirements ,jre for formative evalu- 
ation and the student products cannot be subjected to a summative evaluation 
assessment suitable for an operational training system. Granted, certain end 
product evaluation data will be gathered, where graduates were employed; 
employer evaluations, etc., but the present subjects trained under the pilot 
operational phase of the training program w? ' • t>o*: be representative of what 
the operational training system will produce. 

0. Evaluation Data .and Faculty in the Developmental Training Svstem 

The training development system is also using the present faculty to 
iron out problems in the use of ma'^erials and will hopefully build their 
knowledge of pitfalls and how to surmount them into the operational training 
system. In the same sense r;s the students evaluation data particularly on 
learning product deve lop;nents serves immediate formative evalu.it ion purposes 
so, too, the faculty's evaluation data is simi larly restricted and is not 
an assessment of the operational training system. The evaluation mechanisms 
in points A and B above will gather these data during the coming year. The 
forms developed in the ETS design will be useful during the cperational 
training program, though they should subjected to formative evaluation and 
revised during 1972. 

The pilot operational training program will condition the faculty, and to 
test the operational training program a different faculty should be considered 
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for 1973-7^. 
E. Summary 

The Information --nd requirements for the training development sys- 
tem (1972) are for formative evaluation that will shape the modules and 
engineered internships into an operational training program. We recotT„nend 
specific steps in materials evaluation to refine the instructional design 
and content of the modules and the use of transactional evaluation to structure 
and implement the evaluation data. To accomplish these there -is a need for 
an internal evaluation design. 

' " • How Are The Evaluati on Requirements in the Subsystems of the Training 
Program Being Met ? . 

An analysis was made of the evaluation requirements of the systems of 
the training program with a view to readying them for use in the trans- 
portable train!:-:g trackage. Because the "Training and Development System" 
is part of the present development phase and is evaluated through the assessment 
of the subsystems, it is not included in this analysis. In the following 
tables, each of the separate subsystems object ives were shorthanded from 
the descriptions in SectionNand an analysis was made of (1) the adequacy of 
the data needeH, (2) provision for instr;jments or records needed and where 
called for, (3) a strategy for the use of the data, instruments, and records 
in the training systems, and (k) the adequacy of the evaluation for this 
objective at this time. When examining the objectives, it is advisable to look 
at the full statenents of objectives in Section N of the original project 
description. There is considerable redundancy in the four subsystems. We 
believe careful attention to the objectives of the Personnel subsystem and 
Training Implementation Subsystem in the Evaluation System will cover 
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the significant training features. Thus, we have done a detailed analysis 
and made recommendations on these two. 

EVALUATION REQUIREMENTS 
IN SUBSYSTEMS IN TRAINING PROGRAM 
Table I 

PERSONNEL SYSTEM (OBJECTIVES) 

Data Defined \ ns tramen t (Record ) Strategy Av lequacy 

1. Recruitment X X X YES 

2. Placement in X X X ? 
Program 

3. Programming of X X X ? 
Students 

k. Exit Interviews X NO 

5. Maintenance of X X X ? 
Attendance and 

Performance Records 

6. Personal Counseling X NO 11 
and Guidance 

Recrui tment . The information provided in the three instruments: Application 
Form, Pre-Admission Inten'iew Schedule, Post-interview Rating Form along v/ith 
t* e early interviews on students* reactions to the program will render 
adequate data on recruitment. Taking this information and doing a routine 
analysis ./ill S5^"isfy most of the questions on the individuals who are encouragei 
to apply for the program, those who are accepted, and whether the program 
is meeting its goals of upgrading individuals vocationally. 

Placement in Program . Placement in program has met nost of the four 
requirements with one exception in the area of definition if data. The program 
placement strategy does not provide for using diagnostic tests for deciding 
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which modules that students will enter. The program placement is based on a 
conference report and program plan'. And while the program decision strategy 
calls for cooperation between the Instructor and counselor In defining 
the data for this form. In cases where this collaborative arrangement Is not 
followed, the flexibility of a functional competence program can be lost. 
Furthermore, the record does not allow for recording exempting out of modules, 
or satisfying a requirement In other ways. It is also probably a mistake 
to assume that this record will be a completely accurate program plan, and 
instructions for revision and updating should be Included in the evaluation 
plan. On this subsystem objective. It is reconiP.8nded that placement In the 
program procedures and records be modified to allow for adjustment of 
students- programs on the basis of diagnostic test Information which should 
shape students entry Into and progress through the modules. Instructions in 
updating of the record should be provided. I.e. every six weeks, three months 



etc. 



Programming of Students. This objective uses the Composite Student 
Record which Includes Application For. Pre-admission Interview Schedule. 
Summary of Pre-instructTonal test scores. Syropsis Report of Conference. 
Program Plan. Notice of Completion (of modules, learning experiences) 
as Its main sources of data. Other materials from the Composite Course Record 
and the Composite Internship Record would be Inputs in the student programming. 
It is further recognlrerf that over a period of time follow-up data from 
employers would Influence the student programming through a shifting of the 
curriculum to overcome identified weaknesses. Follow-up data should be 
collected In a systematic, organized form. 

What seems, to be most needed Is a vorm to keep track c?f student progress 
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and it Indicate the students route through the program. It is recommended 
that a form be created for inclusion in the evaluation package that will list 
candidates names on the ordinate and the modules and internship experiences 
on the abscissa. These blocks can be color coded So indicate the adequacy 
of progress and the rapidity of progress for individual candidates. Since 
the progress of candidates is determined by performance rather than time, 
thes-. data should tell at a glance which modules need refinement or redefinition. 
In a similar manner they will inform the personnel staff of candidates v;ho 
need more branching in their programs. Since there will be a quantity of 
information in the Student Composite Record, a form for simplifying and 
drawing the personnel staff's attention to students having difficulty 
in the program is needed. 

While some of these data on student programming could be stored on 
a computer, the scope of the program in the foreseeable future does not 
merit this expense. At this time, we see no advantage to including 
extensive and expensive computerized operations in the data processing 
in this program. The above type of sunwnary form will serve the purpose 
and be more transportable. 

Exit Interview. There is need to define the parameters of the exit 
interview in order to gather data on any attrition in the program. The 
estimates of the parameters of this Interview nseds to be tested during 
the developmental phase of the project (est. 1972-73). It is recommended 
that ETS draw up a student Exit Questionnaire and Interview Schedule. 
Sam Levine's preliminary interview data shoyld be useful in designating 
the first rourh dimensions of this form. 

Maintenance of Attendance and Record Forms . This objective suffers 
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some of the same problems that were covered under the Programming of Student?^^ 
On the v/hole the ETS formulation and categorization of the record forms 
and instruments needed are adequate. The grouping of these data into a 
system of records with three functional referents, Student Composite Record, 
Composite Course Record, and Composite Internship Record appears to be 
adequate as organizers for a wide variety of data. 

What is most lacking is forms for summarizing these data and placing 
them into a graphic display that will encourage the Personnel System staff 
to bring them into the formative evaluation that is to be a part of the 
0, D and E program. The outline of a sample type'form is suggested in the 
Program Planning Section above. It is recommended "that ETS^produce similar 
summary data presentation forms for the other two areas. These forms should 
be used and tested during the development phase of the project this yccr. 
In this way, the usefulness of these forms can be evaluated and the strategy 
for their use be improved. As indicated in Table I this is largely misning 
in the present evaluation scheme. 

Personal Counseling and Guidance . The data is defined and {f gathered 
is laraely adequate for personal counseling and guidance. The records for 
maintaining infonnation on types of guidance sought or personal counseling 
initiated by students and the instructors or the personnel staff are pot 
listed. V/hile the sumrviary data presentation sheets will provide a running 
record on th^ educational guidance, there is need to select a form to maintain 
records of contacts betwee ; the personnel <;taff and students as well as 
summary of progress and attendance. A number of form records are available 
(and they already may be in use), but it is recommended that a standard form 
be selected tested and included in the evaiuatia* oackage as a part of the 
final product. 
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Selection of this form and the development cf forms should be guided 
by concern for future transfer of these data to IBM ca.rds. Experienced 
programmers cat\ gjve suggestions on setting up'the formSf^or this- 
purpose. 

It Is further recommended that the way In which these data may be 
used to assist in formative evaluation of the project be included In the 
nanual in the evaluation package. There evaluation suggestions should be 
collected by the internal evaluator during the pilot operational stage cf 
the one program. 

In Table- I adequacy of the r-easures column is questionable largely 
due to the stage of operation. As the project proceeds and Instruments 
and strategies developed, these question marks should be removed, pre- 
ferably during this pilot operational year. 

Table II 

Training Implementation System (Objectives) 

^ Data Instrument 



Defined (Reco rd) Strategy Adequacy 



1- Staff Snd Adntinistrative 
Preparat ion — InternshI p 

2. Administrative ArranrefT:ents to 
Oversee the Program 

3. Cooperating with ' aboratoi-ies and 
Development Agencies for Training 
and Permanent Positions 

^- Articulating Formal and Applied 
Segments of the Program 

5- Providing Tor Regular evaluation 

6. Altertjig the Program as Indicated 
by Evaluation 



NO 



X 

X 



X 

7 



NO 

7 
? 
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The training implementation syste™„i,, be largely formulated in 
the pilot operational year, particularly i„ the shaping of strategy for 
administrators to implement the objectives. Thus, the last two columns of 
Table 11 are mostly question ™arl<s. One major focus for the internal evaluator 
"ill be to gather data on how strategies are evolving on use of data to 
implement the objectives of this system. Some specific suggestions for 
each of the objectives follows. 

StafL and Ad.inlstrarlv. Preparer ion-, nte rnshj^. The applied asoects of 
the pro,ra. contained in the internship wi ., require the overall administration 
of the program to use i.plenv.ntation strategies which will provide training 
for those supervising the internships. The initial data collected on the 
backgrounds of the supervisors of the internships should give an indication 
of the aoK^unt of training needed. Additional information which will be 
specific on training will be provided through the assessment interviews 
periodical ly conducted, the Intern-s Descrlpt ion -of Inte.-nshlp, Internship 
Rating of Internship. Supervisors Rating of interns, and the personal 
counseling data. A strategy for the use of these data is suggested in the 
section of Transactional Evaluation, sec Page 9. It is reco.«^nded that 
strategies should be tested and procedures for t.heir use described in the 
operational training package during 1972. 

MlIIMijsXrative_^^ to Oversee rh. P..^..^ jhe efficacy of 

the administrative arrangements has been difficult to assess. In the progress 
of the project thus far the administrative arrangerr.nts have been taxed by the 
start up pf,ase of the project, which is not a valid test of the ad.-ninis- 
trative arrangements for the operational training package. One factor is 
becoming clear, however, and that is the need for coordination between the 
three main units of supervising agency, formal teaching, and internships. 
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So far the input back into the system from evaluatiofi data has not been 
sufficient Moreover, pressing administrative details on development have 
minimized the role of evaluation. One of the most important questions on 
this objective is whether the administrator can also function as an 
interncil evaluator who handles the formative evaluation of the project. 
Some data may be gathered on this question in the present year. The answer 
on number of administrators needed and how they devise strategies to meet 
the demands of the systems managers for accurate evaluation that can be 
incorporated quickly into the program may need to be explored in depth 
when the operational unit is completed. We do recommend that a systemat;. 
interview shcedule, once every six months, be used v/ith the present 
instructors and participants, along the line of Sam Levine's interviews. 
The data bearing on this objective should be extracted and stu died for its 
implications for administrative arrangements. Answers to such questions a-, 
can the course module instructors function as supervisors of interns, or 
must supervisors of interns be in the coordinating agency, are of primacy 
in completing the operational package. 

Co operatinq With Laboratories and Development Agencies for Training 
an d_ Permanent Positions . This objective was explored in a prelinjinary 
fashion in the original proposal. There are some indications that there 
is a need to further refine these data. How the purchasers of the training 
package would conduct a survey, what specific agencies would be included 
and the alternative routes that an internship n.ight take are cloudy at 
this time. The training package should have specific suggestions to make to 
the user. Also the internal evaluator will need to follov/ closely the 
internship experience to document the pitfalls that a central coordinating 
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agency should avoid. For example, we found there existed considerable con- 
fusion on the part of intern supervisors on what they should do with interns. 
From their remarks frustration was building with the program, a condition 
which will be reflected in the relationship between the supervisor and 
intern. Another area where the Training Implementation System needs 
broadening is in the extension of the job opportunities to other agencies. 
If this is not done, we believe that graduates of the program wiill have 
difficulty in securing positions that utilize their training. Certain 
research evidence would point toward more effective placement if intern- 
ships were done in the same place as permanent employment is sought. We 
recommend that more evaluat.ve data be gathered and analyzed on the intern- 
ship using the present forms with an eye to including specific details on 
its implementation in the operational training package. Also we recommend 
that the present objective be re-examined on the basis of its-original 
assumption and a wider range of agencies be investigated for intern 
placement. Further we recommend that the use of transactional evaluation 
as a strategy with intern supervisors at agencies to improve the formative 
evaluation of the program, and strengthen the coordinating function of the 
Training Implementation System. 

Articulating Formal and Applied Segments of the Program . The data 
which are gathered by the suggested forms developed by ETS seems Co be on 
target. While the three major areas of data wi 1 ■ provide sufficient 
information, the use of these data in a formative evaluation strategy is 
still to be resolved. The supervising agency (FWL in thir, case) needs to 
explore strategies for articulating segments of the program. We have 
recommended one through use of transactional evaluation. Other strategies 
could be tested. The main outcone is to put these into an operational 
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training package which direct the purchaser in articulation of the program 
and assist him in avoiding some of the pitfalls that overwhelm programs 
undertaken by consortiums. Much of the data will be gathered through the 
present forms, the strategies for their effective use are in need of 
being developed and tested. We found little evidence of this problem 
being handled either by the administrative staff at FWL or by ETS. We 
recommend that an internal evaluator be used to test strategies and 
evaluate the adequacy of the data being generated. The evaluation of 
the articulation of the program should entail an ^analysis of the curriculum 
materials as well as student and faculty behaviora I, data. * 

Providing for Regular Evaluation of All Segments . The present 
evaluation plan is deficient in being an integrated whole which can be 
installed and operat ional ized by a user of the operational 0, 0 and E 
training package. The packet of evaluation forms developed by ETS are a 
start on a ready made evaluation package. We recommend that these be 
completed, tested, refined and put Into an evaluation design that suggests 
what the purchaser do on implementing evaluation when he uses the training 
system. The evaluation of the pilot operational phase of the package and 
the evaluation for the operational training package, as we have stated in 
other places, are two different problems. There m?y be a need to simplify 
the evaluation and use a fev; indices, if the training package is to be 
adopted to a variety of contexts. The essential evaluative data should 
be identified during the pilot operational phase, 72-73. Equally as 
important is the outlining of strategies, which should be tested and 
refined to be included in the package. We have made numerous suggestions 
on both these points throughout the document. 

Altering the Program as Indicated by Evaluation . The use of form^ative 



evaluation strategies and their relationship to the data gathered by the 
proposed instruments will be a significant part of the finished Training 
Implementation System. We have made recommendations in the other 
objectives of this section which pertain to this objective. Again the 
separation of the present work in foimative evaluation for.. the pilot 
operational stage from the evaluation requirements of the completed 
training package will clear up much of the present confusion on what an 
evaluator should be doing. Evaluation data at present are not guiding 
and shaping the program the way it can and should. Our recommendation on 
how this situation might be corrected is spelled out in Section II of 
this document. 

'V- Some Spec ific Recommendations on the ETS Evalua ti on Design and 
InstrufTientation 

K The basic assumptions underlying the model for evaluation 
proposed by ETS are conventional and generally accepted by evaluators. 
Where the model may encounter difficulty is in its use with the pilot 
operational phase of the project. As it exists it speaks more to gathering 
information and stimulating decisions on an operat ional ized fully 
integrated D, D, and E program. 

At this point in the evaluation there Is a need to field test the 
forms that have been developed to determine: (1) if they gather information 
in a systematic reliable way, (2) if they gather data in a form that it, can 
be readily utilized (format ively) by the participants, (3) and if the data 
gathered will assist in summative evaluation of the project. 

It is recommended that ETS continue to refine the Evaluation Design 
and to field systematically test the instruments during the developmental 
phase. From these efforts a package of field test instruments and a 
written manual for their use should be included in the operational package* 
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2. 'Student Progress Information, pp. k^S, Is based on a comparison 
of actual progress with postulated progress in the previously prepared 
program plan. Again this evaluation should be more related to the pilot 
operational phase of the work in 1972- It is highly unlikely that the 
personnel staff and student can predict with any degree of accuracy the 
pace of learning or pitfalls in an untested program- However, the data 
on student experience with modules will be useful in providing a basis for 
estimated completion time, recycling and branching needs. It is recommended 
that during this first year, careful records be made of student and 
instructor reactions to the course materials. This requires corsistent 
monitoring and a feedback mechan 1 sm for this information. There is at 
present structural limitations in carrying out this evaluation role which 
needs to be corrected, probably through the appointment of an internal 
evaluator* 



5» P. 6, How will data on actual performance in the classroom and 
internship be gathered? Who will assume responsibility for it? Will the 
instruments simply be administered by the personnel staff? Will an outside 
evaluator or ETS use these instruments on a test basis? 

^. P. 9. The direct ratings of the course will produce data on 
gross perceptions on the course by both students and teachers. There is 
a need to obtain more specific monitoring information on hpw the modules 
fire being used. These data- can be gained through careful Interviewing 
or visitation of classes and internship sites. Who will do these? Will 
this be part of the data gathered in the interviews? Will ETS perform 
this role? How wi 1 1 this role be described in package? 
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5. P. 11. Can the overall systems evaluation wait until groups of 
students are turned out and employed? Would it not be possible to do some 
simulations with an evaluation team extraneous to the project to obtain 

a reading on students proficiency? 

Would it not also be important, despite the students' experiences, 
to follow-up more than one year? Could not data on the context of 
graduates' experience be included? Also the mapping of the career paths 
for 0, D, and E products would be an important part of follow-up. 

6. P. ]k. In the form.itive evaluation of the present program, a 
comparison of the model should be made with other programs of a similar 
nature, e.g. University of 'Virginia, masters and doctoral programs 
purporting to prepare candidates with similar skills. 

The appendices referred to in the above report are not included with 
this report. 
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n. ACTIONS REGARDING RECOMMENDATIONS OF EASH-WALBERG EVALUATIONS 



In revising and improving the evaluation system, ws have taken into considera- 

lon the recommendations on pages 27-29 of the Eash Evaluation RepSr^ Ihe^e 

were six specific recommendations contained in this report. Gur actions in 
regard to each are outlined below. dcnunb in 

ihrouSh flnif f!o'^'?" '^'i-"" ''^''"9 "^^^^""^^ the instruments tested 
through field use. Consortium personnel have substantially revised these 

ReoorJ nr cTnl f requirements included in the original Design 
PvML;nnJ f ^•'"P «^ ETS) who was involved in the initial 

irih?s act^iuy*!' ''''^'"'^ °' ' consulting basis to assist 

2. We are now maintaining records of student and instructor neactions to the 
SnTtes eS^'^M- '^'''^'1]'^ feedback system for accomplish] th s s 
.^^Jnnln ^hf" '^r:/''^^^'^^'? Elzey. a member of the Consortium staff, has been 
de Snn . 5°''*!°" °rj"ternal evaluator with primary responsib 1 ity for 
develop ng and implementing evaluation procedures. His immediate concern 
IS Obtaining information related to th^ pilot operational phase this year. 

LmaT °2ir^?n!r°'" Prf'^^"i^"".is the responsibility of the instructional 
manager. All instruments pertaining to student performance will be admin- 
t- evaluated by each instructional manager prior to submission to 
Consortium management. Data on intern performance is the responsib 1 ity of 

on a task by task basis. The advisor eval-.^ates intern performance by re- 
l^ecSrd^f^l'e'"' '""^ '"^ the intern's 

b; thplnL'n^lT ?^ l^^ courses will be derived through interviews conducted 
intfJncM^ f evaluation staff (see #1 above) and through visiting classes and 
Sh?JifhLV'J!'- ,^\Pl°:^tory discussions are being conducted with Dr. Danie? 
a clnler ?n;p?S'° Evaluation Center, for consulting services and placement of 
hp.n ?n J"!^? ^""^ West Consortium. Discussions have also 

been initiated with Dr. Michael Scriven. UCB. for consulting services. 

5. We are attempting to implement evaluation procedures as various parts of 
the system become operational . Follow-up information, providing feedback on 
effectiveness of Consortium materials and program will be continued' as long 
as the Consortium is funded and the gathering of information cost effective. 

6. We do not anticipate comparing the Consortium program with the other 
irodeis of a similar nature. The Consortium will review the operations and 
...cuerials of these models that might meet Consortium needs. However, the 
Consortium model ;nust stand or fall on the effectiveness of its own prooram 
in meeting development agencies' on-the-job requirements. Procedures to 
provide this information have been established through the engineered intern- 
ship and graduate placement. 
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FAR WEST LABORATORY FOR EDUCATIONAL RESEARCH AND DEVELOPMENT 

III. INTER-OFPICE MEMORANDUM 

Dote July n. 1 972 

n r> 1 , „ . '*3y"e Rosenof f , JoeJ alard 

To: Bela f^, Banathy cc: Paul Hood .HT=?^5^n~fTyiu- 



Paul Hood^^HFfeetaan fT^^>Oohn Heurlgan. - 
From: -y^^ D^- Maurice J. Eash. Director. Office of EvaluItT^F^i^ch , 

. y~ ^ University ot Illinois at Chicago Circle " 
Subject: u.^ D&E Proje'^t, Evaluation Consultation — July n. 



On July n I reviewed toe progress of the evaluation design f:r the D,D&E project 
with Mr Freeman Elzey, the internal evaluator. This review was a follow-up of 
the evaluation consultation completed in March 1972 where a number of recommendations 
were made for strengthening the development of an evaluation system. InX^reWew 
the evaluation design and instruments were examined and a number of verbal 'sua- ' 
gestions. made. 

The four major recommendations which were the core of the March report have been 
d2?MoSr?*c'"5 ^valuator has made excellent progress in meeting the ■ 

deficiencies of the evaluation system. At this time, the conceptual framework 
tor guiding the final development of the instrumentation is rapidly emerging, and 
the preliminary instruments for the training package are in a rough dr^'ft stage. 
Whan the conceptual framework is completed i., the next few weeks, the '3rtstrument.«: 
can be rurther refined and will be ready for audifional field testing. 

Plans are well undemay for the development of manuals to accompany the instruments 
in the training package, and the procedures for processing and analyzing the 
data for foiTnative evaluation purposes are being refined. 

It is my professional judgement that an evaluation design is', emerging that will- 
provide formative evaluation data on the development of the training package 
modules and their implementation, and (2) produce a series ef usable evaluation 
Sc"'"^?^ ^ procedures to accompany the training package. The future evaluation 
needs of the project appear to be well in hand and the PERT c;-.art for meeting these 
needs IS reasonable in its demands and deadlines. The reorganization of functions ' 
into four areas has strengthened the project's evaluation component* and the 
internal evaluation i-. building an effecti-ze and efficient evaluation system. 
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IV. EVALUATION ACTIVITIES DURING FIELD TESTING 

1972-73 



A Report For The 

TRillMING PROGRAM FOR DEVELOPMENT, DISSEMINATION, 
AND EVALUATION PERSONNEL 



April 1972 
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RATI0NAL3 

The necessity to evaluate 60 modules — 30 at each 
professional level bciin^^ trained — ^ In Che flvft courses being 
field tested in 197.V-'>'1 dr.rn^inds th^ r:r]ort5> of tne evaluator 
be sharply focussed. In essence, each of the modules is a 
mini-course. With a ralnlmum of two working days devoted to 
each module — a day of prepriration and data collection during 
an evaluation session qvA a day of nieraorandura writing per 
module — the evaluator hcs utilized ?0% of the available 
time during the academic year. Three visits to interview- in- 
terns and write the reports necessary will utilize the re- 
maining 30 % of the time available. 

Preparation for evaluation prior to September 1972 and 
follow-up activities during June 1973 utilizes the remaining 
time of the evaluator for the field test year. Preparation 
prior to September 1972 includes reading and studying the madulea 
sufficiently to be able to frame some questions related directly 
to each and to be able to intelligently dlsousR the content 
with students, writing individualized evaluation ••inserts" to 
accompany modules as they are being sent to instructors for use, 
and arranging for cooperation during the academic year with both 
Instructors and internship agencies. 
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a2co-^:ii':NDATrciis 



It Is reoommend^d that the Internal ftvaluator concentrate 

his evaluation efforts during 1972^73 on the follo^^ln/? essential 

activities! 



(1) the arran^ln^:, scheduling, and conducting 
of evaluation discussions with students, 
Instructors, and authors in sessions aimed 
at obtalnlni^ negative feedback on each 
module used during; 1972-7?; 



(2) the arrancrin^, scheduling, and conducting 
of evaluative interviews with all int«5ms 
and their supervisors on-site in their 
tralnlno: a^-«ncies at periodic Intervale 
throughout the year but with more contacts 
during the early months of the experience! 



(3). the preparation of evaluative raemoranduraa 
to the developers of each module listing 
the major flndln>?a and coinraente the evalua- 
tor feels are most significant for prepara- 
tion of a revised module; and 



(^) the preparation of three evaluation reports 

on the Internship situation — site by site 

approximately by Thanksgiving on initial 
conditions and problems, by February 15, I973 
on interim conditions, and by June 1 on the 
total Internship situation and recommendations 
for changes by administration for 1973-7^; 
interns. 
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PROCEDUaSS 

AS soon as puiialble after completion of efforts to «raock-up* 
evaluation data for 1971-72 prototype course modules and get 
this to the developers a the Internal evaluator should turn to 
the preparations for 1972-73. iln any case. It will be wise ta 
have a deadline for this year of June 30, 1973_ ^nd to focus 
attention after that time on next year's requirements which 
are extensive. 

Starting with the available units and adding the remainder 
as they become availalile during the aummer, the evaluatcr will 
review each sufficiently to develop an "insert* memorandum to 
the students that will be using the module. This Insert will 
be Individualized enough for the unit so that certain aspaots 
or kinds of information eventually to be discussed during the 
evaluation session will be made clear to the students. That lo, 
they are to be alerted to the fact that an evaluation session 
is going to be scheduled for the module they are learning from 
and to the kinds of information that they should be making nates 
about or thinking about while working through the module. Thl» 
is not as dsstftiled as a questionnaire on the module but will requira 
thought and careful development by the evaluator. When it is 
ready, it can be circulated for comments or additions to the 
developer (perhaps) and then printed up and inserted in the modules 
ready for delivery to the instructional managers. 

Development of a master schedule and calendar of evaluation 
activities — based on a realistic school year calendar — may 
begin during the summer although it will be changed and completed 
as the academic year proirresses. 
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Because the cooperation of instructors and Internship ajrencles 
Is essential to the evaluation activities plonnert, some summer 
time should be devoted to gettln/i; to know the individuals involved 
and In discussing the evaluation arrangements envisioned. To the 
extent that the Internal evaluator already is acquainted with the 
Instructors or internship supervisors the task will be reduced. 
However, for those individuals that he is not acquainted with It 
will be Important to schedule one and half to two hours of time 
to discuss the purposes and procedures of the evaluation activities 
to take place during the acaderalo y«ar. If most of this ground wor 
can be accomplished during the summer, the actual academic year 
should move much more smoothly and with greater coopemtlon. Prior 
knowledge of the evaluation activities and time to adjust to the 
idea that such activities will occur reduces a frrep.t deal the na- 
tural and human anxiety of those asked to cooperate. (Rapport 
with the students asked to evaluate the materials is essential also 
obviously. ) 

If the evaluator is planning to utilize an assistant for a 
portion of the evaluation sessions with instructors or when 
interviewing the Interns, this person should be Included in 
the summer activities laying the ground work for data collection 
during the year. 

As the Fall Semester begins and the first modules are being 
delivered for use, the evaluator will need to confirm times, 
places, and dates for the evaluation sessions by contacting 
i;he instructors.. If the dveloper is going to be present for 
the evaluation session, coordination with his schedule will 
require further communications. In any case a person or a 
raeohanloal device (like a casette tape r^corn^r) should be 
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utilized to record co:r .«^ncc ' Wi llift se;^:ilons» This must ba 
planned for and arranged also. 

(A Generalized Forrnulatlve Evaluation for .\o»;uLft:. ». .'^ 
described elsewhere and may be useful to Indicate how these 
evaluation sesslonsmay proceed.) 

Following the Information collection In the evftluatlon 
sessions, some of the most difficult and Important work of 
the evaluator takes place. The evaluator must prepare a 
brief, but detailed, memorandum, and accompaiylng documents to 
attempt to help the author of the module to consider specific 
places or ways to revise the module studied. This memorandum 
is the heart of the evaluation process because here the ideas 
and su/?sestlons .mentioned at various places and times or 
emerging from the flow of information generated by the ©valua- 
tors questions 'Will be summarized and recommendations presented. 

The accomp.^.nying documents may be students* and instructor's 
copies of the module with notations and questions written in 
the margins — or summaries of the comments recorded by the 
tape or by an assistant acting as recorder or written comments 
from the instructor. . In all oases .the evaluator must take the 
responsibility to attempt to summarize what the information means 
to the developer — even if the developer was present in the 
evaluation session. In this way the evaluation function is 
fully operative. Without this attempt the evaluator merely 
becomes a channel through which Information flov/s but Is loft 
untouched and no evaluation has taken place. 
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Also stflirtln;; in the Pall Semester £\re the Interviews with 
Interns and supervisors on-site to evaluate the situation for 
each Individual trainee* During each Interview session there will 
be time for separate discussions with thfi Irtern and the super- 
visor as well as a group meeting* The Individual sessions can 
precede or follow the group discussion according to the desires 
of the participants In each site* The purpose will be to collect 
both descriptive Information about the experiences actually being 
provided as well as reactive or affective response to the Intern- 
ship situation* 

Since each situation Is Intended to be Independent , It does 
not seem possible to merely sample what Is happening to Interns 
as a group and arrive at any helpful comments* The Intention of 
the evaluation Is to find what experiences In what kinds of situa- 
tions are helpful what kind of students achieve the goals of the 
the Dt D & E training system* Because this Is so, the evaluator 
will need to summarize the Information collected site by site 
In order to make recommendations to administration about changes 
In the Internship program as established* It also requires that 
sufficient time be devoted to each site to allow a full and de- 
tailed discussion to develop about what has been happening to 
the Intern* It Is possible In some situations that the Intern, 
the supervisor and the evaluator will go over certain paper work 
that the Intern has been Involved In on a page by page basis In 
order for the evaluator to understand what was happening and to 
Judge for himself the value of the experience given the goals of 
the training system* 
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Th-j reports prepared Ly the evaluator at three times during 
the year on the Internship situation should uiroucea at che 
Administrator of the Internship program Internally within the 
Par './est Laboratory, attempt to reach general Izat Ion but pre- 
scsnt recommendations In terms of specific situations, and, of 
course, avoid the use of locatlonal or nominal identlf Icatlone 
In all permanent printed oommunl oat Ions. The ©valuator qua 
eyaluato ^ r Is not Interested In the defects of Mr. Jones and 
Mrs. Smith but only In the essence of the Internship eiperlenoes 
as they are developing within the training system. This is a 
difficult but essential distinction" that must be maintained if 
the evaluator is to secure and maintain cooperation during the 
course of the year. 

General 

The two major assets of any evaluator /\re the ability to 
form probing questions that uncover information of potential 
value in forming judgments about Important aspects of any sit- 
uation and the ability to Judge — ths.t is, to take a position 
based upon the Information collected. The intention of the 
recommendations made in this report have been to clarify the 
opportunities that the ©valuator has in the Training Program 
Field Test Year to perfect and improve the system by the timely 
and continuous collection of information. Within the limita- 
tions of time and personnel and — judgment — , the evaluation 
activities outlined here should benefit the Training Program 
substantially during 1972-73, 
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Appendix G: LEVELS OF COMPETENCE 



and on the amount of opportunity he/she has had for app vino It in thp 
de?l:ft,-:n' miA'^''^ WveJs^^Jf S^^t^Sce'lnd a 



LEVEL 



ORIENTATION 



FAMILIARIZATION 



LOW PROFICIENCY 



HIGH PROFICIENCY 



DEFINITION 



The task has been described or demonstrated; 
intern understands its purpose or function 
but cannot perform it. ' 



Intern has been given practice in performance, 
but can perform only with close supervision or 
detailed instruction. 



Intern has been given repeated practice. 
Performs slowly with few gross errors, if 
given some supervision or adequate job aids, 

Intern performs efficiently and with no 
errors. Minimal supervision required. 



Appendix H: PARADE FORMS 
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NCERD Reporting Form - Developmental Products 



1. NQm« of Product Functional 
Competence Training Model for 
Development .Dissemination, 
and Evaluation Personnel, 
Series 1: Planning and Design 



2. loboratory or Contor 

Far West Laboratory for 
Educational Research and 
Development (FWLERD) 



3. Roport Proparotion 

Date prepared Jan 10, 197 3 

Reviewed by Laboratory 
Director 



4. ProbUmt Decoription of the educational pvoblm this product desioned to solve. 

There is a major demand for personnel who are qualified to perform at various professio 
levels in the field of educational research development, dissemination, evaluation and 
implementation. Surveys of current training programs for developers indicate that 
the output is miniscule in relation to the demand ♦ 

In the area oi planning and design, the need is particularly critical. Through 
literature survey and analysis, time sampling, and task inventories, it was determined 
that planning and design are activities in the D,D&E process which can be performed by 
.persons with appropriate training at the entry professional levels. 



5. StfaUgyj fhe general strategy 8elect:»d for the solution ?f the problem above. 

An approach i^ivolving job and task analysis, specification of behavioral objectives, and 
forTiative devvslopment of materials and systems was selected as the general stratc^gy. The 
dei;ign requirements included: a resource- ,^ased, functional competence approsch; multi- 
level prograii^ming, modular instructions, units, i. learner-centered and learner -active 
method , trap ^portabili^'y and for long-term cost-effectiveness. 

Planning and Design were judged to be high priority in development. The development 
emphasizes teaching these activities in a functional, task-oriented setting. 



g 



6. Reloaso Ootot Approxzmtc date 
Droduct \*as (ov vilZ br) rear..* 
for release to n-.-i agency. 



Second Quarter 1974 



7. lovol of Dovolopmont: Cnaracx^ »- 
Latlc level (or i^zo^'ectcl lave, 
of dp,velopment of rrcduct at ti- 
of release, Oiec< one. 

l^podu for critical review and for 

pvcjaratic* for Field ?est 
protc^titpc ruxter'-xir* 
^ Rcadj for Field Tost 

r.cady for publisher modification 

if<5aJ. for general disser:l*:ation/ 

iiffusion 



8. Noxt Agoncy: 

rrrduct ix:.' 



pi 
'ri> 



mjo*:ci 



'js^a 



10.71.A (D) 



■ ' ^ 125. 

. P/oduct O*«cription. D^ccribe the follouing; m^er each deeoription. 

• Z. Characteristioa of the product. • 4. Ataooiated ptvduot^. if any. 

• 2. Hod it uorhe. 



♦ 3. What it ia intended to do. 



• S. Special conditions^ time, training, 
<iquipment and/or oth*>r requireirente 
for ite uae. 



. The Planning and Design series is a skill-oriented, job-related, training 
onsisting of modules that permit flexible scheduling by both individuals and small 
roups. The series is largely learner-.nanaged and emphasizes personal interaction, 
mmeaiate and frequent feedback » and special student activities. The series will 
rovide instructional experiences designed Zo develop specific skills in finding the 
est way to plan and design a product or process to meet its established objectives. 

The series will be implemented in three phases: a preparation phase, an intensive 
earning phase, and an application phase. During the preparation phase, a candidate 
or training receiver detailed information about th-^ training and has a guidance 
assion with the Instructional Resource Manager (IRM) . He then takes a pretest to 
etermine if he can profit from t^s course. If he is accepted, he proceeds into the 
ntensive learning phase. The trainee then works with a set of learning episodes 
xthm a module focused on a specific skill. Frequent self-evaluation checkpoints 
nd counseling inform him of his progress. The modules provide simulated applications 
t the target skills. During the application phase the trainee begins an engineered 
nternship which may operate concurrently with the learning phase. The trainee 
avelops a plan for work that he would like to execute at a D,D&E agency. The 
upervisor of the trainee, the trainee, and his IRM negotiate the plan and set up the 
nternship. The engineered internship makes possible the specification of job 
asponsibilities and assignments, establishes performance check points, and provides 

functional situation in which the trciinee applies the skills he has learned, ="d 
cquires new ones under realistic conditions. 

. The Planning and Design Series is intended to develop compentence^ in (a) the 
urmulation of performance specifications; (b) the selection or planning of alternative 
pproaches and means leading to the desired performance; (c) the analysis and selection 
£ the most promising alternative (or the design of a new alternative); and (d) the 
reparation of plans for development, evaluation, and dissemination. 

. This series is related to an', can be linked with the Data/Information Collection and 
naiysis series; it will be recommended that they be used as concurrent series. 

. Approximately 8-10 hours of work will be required to complete each module in the 
eries with fractional units of credit assigned per module. 

he series also requires an Instructional Resources Manager who will coordinate 
acilities, materials, and student interaction. He will also serve in a resource and 
valuation role when needed. 



ERLC 



126 ^ 

10. Product Us«rs: Those individt^?.l8 or groups expected to use the product* 

Preservice and inservice D,D&E personnel and continuing education students are 
expected to use this series* 



11 Product Outcomes: The changes in user behavior, attitudes, efficiency, etc. resulting 
from product use, as supported bii data. Please cite relevant support dccunents. // 
claims for the product are not yet supported by empirical evidence please so indicate. 



This series is still in early developmental stages; only limited data on 
its effectiveness are yet available* 



12. Potential Educotionol Consequences: Discuss not only the theoretical (i.e. conceivable) 
implications of your product but also the more probable implications of your product^ 
especially over the next decade • 

Manpower supply and economic reality require that development work be accomplished 
with relatively large ratios of entry-level professionals* The D,D&E program in 
general and this series in particular is aimed at preparing personnel on the entry 
professional level to perform tasks on the lower levels of development, thus freeing 
the highly trained or more experienced developer for work on the more advanced aspects 
of D,D&E. This situation will be both more efficient and more cost-effective* The 
demonstration of an economically justifiable place for entry-level professionals 
represents a challenge for the Far West Consortium's design, as employers have not 
traditionally seen a need for specially trained personnel at this level* However, 
it is hoped that a successful program will lay the ground work for further training 
and emplo3aiient* 
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13. Product EUmtnts: 

Hot th(> elements Sni^^h constitute the pvodua:^. 


14, OriQIII: 

Circle the nost 
cppropvia te letter. 






A<;<jessraent Instruments 


D M A 


Module 1*2 Consideration of Alternatives 


(D \ M A 


Assessment Instruments 


D xV A 


Module 1.3 Planning for Development 


(Sjf^S A 


Assessment Instruments 


£> M A 


MoHiflfi Planning for Evaluation 


(D)MA 


Assessment Instruments 


£> :V A 


Module 1*5 Planning for Dissemination/Marketing 


t u : ly A 


•Assessment Ins truments 


D X A 


Module 1.6 Introduction to Component Design 


^ — ^ 


Assessment Instruments 


DMA 


"~ ' ■ ■ • 


D iV A 




tf fj A 




D M A 




D= Oevclcped 
M= Modified 
A- Adopted 



15. Start.up Coitfi j^^^i ^p^^ted costs to procure, 
insitzll and initiate use of the product. 

Estimated materials cost $4-5 per module, 
including assessment instruments. Other 
support materials up to $10 (pro-rated) 
pc'^ series. Staff training time 2-3 days. 



16, OperotinQ Costs: Projected costs for conziKuiny 
use of produa:: jfter initial adoption and 
installation (i,e,,feeSy consiaiahle supplies, 
special staff , training , etc.). 

Not known at this time; however, it is 
anticipated that the cost will be roughly 
equivalent to that of a three semester 
unit university course. 




of 
d 



Entry-level professionals with the skills covered in this program will be in great 
demand by those employers with the largest (and usually high priority) needs: e.g., 
development programs and projects of educational laboratories, R&D centers, 
universities, non-profit agencies, and industry and business. Certainly federal 
state, intermediate, and local educational agencies will also provfde some employment 
opportunities, but probably to a smaller extent. 
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NCERD Reporting Form - Developmental Products 



114203 
ID/C 



1. Nam# of Product Functi onal 
Competence Training Moael for 
Development, Dissemination and 
Evaluation Personnel: Informa- 
tion/ Data Collection & Organi- 
zatio 



2. Laboratory or Center 

Far West Laboratory 
for Educational Research and 
Development (FWLERD) 



3. Report Prep^rotion 

Date prepared January 10, 1 9 ^ 

Reviewed by 

Laboratory director 



atTorr » 

4. Problem: Description of the educatio>ial problem this produot designed to solve. 

There is a demand for personnel who are qualified to perform at various professional 
levels in the field of educational development, dissemination, evaluation and^ 
implementation. 

In the area of collection and organization of data and infomation, the need is 
particularly critical . 



Strategy: fke general strategy selected for the solution of the problem above. 

An approach involving job and task analysis, specification of behavioral objectives, 
and formative development of materials and systems was selected as the general 
strategy. The design requirements included: a resource-based, functional com- 
petence approach; multi-level programming, modular instructional units, a learner- 
centered and learner- active method, transportabiVity and for long-term cost- 
effectiveness. 



Collection and organization of data and information were judged to be high 
priority in development. The development emp.')asizes individualized instruction in 
a functional, ta^k-oriented setting. 



6. Release Oatei Approxirnate daii 
product vas (or he) reai 
for yeleasc to n:rt agency. 

Second Quarter, 1974' 



7. Level of Development: Cnaract^ : - 
i'Hla level (or pro^'ecte.! level 
of development of product at t'i- 
of release.. Check one. 

iyecidij for critical review and fo'i 

preraratioK for Field Tczt 
'Y.u. prototup^: mater^*al'! 
P .cai:i for Field Tect 
is^cady for publisher modification 
' '.^ggj for general disser:>.ation/ 
i'lf fusion 



8. Next Agency: Ajckqi 
p*cduct DC.-' (cr *y 
W'. 'jsed fr^ fu-»* , 
" ' Vtf Lopmen r. J '/* •* tp 
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9. Product O«*crip«iont Describe the following; number each dessHption. 

•I. Charaateristios of the product. •4. Aoeooiated products, if any. 

• S. Special conditions, time, training, 
•' 3. What it is intended to do. equipment and/or othov requirements 

for its use. 

1. The Information/Data Collection and Organization Series is a skill-oriented, 
job-related series consisting of modules that permit flexible scheduling by both 

iStlrnpf^nLi"^^ The series is largely learner-managed and emphasizes 

interpersonal interaction, immediate and frequent feedback, and special student 
activities. The course will provide instructional experiences designed to develop 
specific skills in collecting and organizing information and data. 

2. The series will be implemented in three phases: a preparation phase, an intensive 
learning phase, and an application phase. During the preparation phase, a candidate 

L [u^^t"^ receives detailed information about the course and has a guidance session 
with the Instructional Resource Manager (IRM). He then takes a pretest to 
determine whether he can profit from the course. If he is accepted, he 
proceeds into the intensive learning phase. The trainee then works with a set of 
learning episodes within a module focused on a specific skill. Frequent self-evaluatior 
Checkpoints and counseling inform him of his progress. The modules provide simulated 
on-the-job applications of the target skills. During the application phase, the 
trainee begins an engineered internship. The trainee develops a plan of work that he 
vvould like to execute at a particular DD&E agency. The supervisor of the 
trainee, the trainee, and his IRM negotiate the plan and set up the internship. The 
engineered internship makes possible the specification of job responsibilities and 
assignments, established performance checkpoints and provides a functional situation 
in which the trainee applies the skills he has learned, and acquires new ones on the 
job. 

3. Upon completion of the series the trainee is expected to have acquired varying 
degrees of proficiency in (a) processing and organizing data from observations and 
interviews; (b) preparing and using visual data displays; (c) data analysis .by tally, 
coding, scoring, etc., and (d) computing common summary descriptive statistics. He will 
also develop his ability to conduct information searches, using resources such as ERIC, 
CUE and libraries, and to organize collected information into appropriate classifi- 
cations within problem areas; to abstract relevant information on specifics within 
problem areas; and to present information in the most appropriate form for specified 
purooses and audiences. 

4. This series is related to and can be linked with the Planning and Design and 
Communication Skills series. These series jointly constitute a significant cluster 
which provides foundation skills for the remainder of the program of instruction. 
Opportunity for application and practice of the competences taught in this series has 
been designed into every other series in the program. 

5. Hours of instruction are used only to convey a concept of series duration in 
convKiitional terms; the series involves individual or small group work rather than 
formal classroom sessions. Actual hours of student work for this are estimated to be 
roughly equivalent to a 3-credit conventional graduate course. The series requires an 
Instructional Resource Manager to coordinate facilities, materials, and student 
interaction, and to serve in a resource and evaluation capacity when needed. A film, 
fimlstrip, slide-tape lesson may be viewed if equipment is available. 
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10. Product Us^rit Those individuals ^or groups expected to use the product. 

Preservice and inservice D,D&E personnel and continuing education students are 
expected to use the series 



11. Product Outcom«t: The changes in user behavior, attitudes, efficiency, etc^ resulting 
from product use, ac supported bxj data. Please cite relevant support documents. If 
claims for the proSuct are not yet supported by empirical evidence please so indicate. 

The series is still in the early stage of formative development; as a result, 
only limited data is available as to its effectiveness • 



s 



12. Potential Educational Contequ«nc«tt Discuss not only the theoretical (i.e. conceivable) 
i^Tiplioations of your product but also the more probable implications of your product, 
espeaially over the next decade. 

Manpower supply And economic reality require that development work be accomplished 
with relatively large ratios of professionals.' The D,D&E program in general and this 
series in particular is aimed at preparing personnel at the entry-professional level 
to perform tasks at the beginning levels of development, thus freeing the highly 
trained or more experienced developer for work on the more advanced aspects of D,D&E. 
This situation will be both efficient and cost-effective. The demonstration of 
an economically justifiable place for the professional represents a major challenge 
for the Far West Consortium's design. However* it is hoped that a successful program 
will lay the groundwork for further training and employment. 
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13. Product Elements: 

iijt ihe elencnts 'hinh amstitute tke product. 


14. Origin: 

Circle the nost 
appropriate let'jer. 


Mobile 2.1: Orientation to Collecting and Organizing DD&E 




Information and Data and Assessment Instruments 




Module 2.2: Observing and Interviewing and Data and Assessment Instrument 




Module 2.3 Dat3 Management and Data and Assessment Instruments 


@li A 


Module_2^4_. Data Analysis and Dat;^ and AQ<;pc.monf Tn<:f„rmonf c: 


(3) M /. 


Module 2.5 The Retrieval of Information Using Bibliographical Sources 


(D)y. A 


and Data and Assessment Instrumpnt*; 


D a 


Module 2.6 The Retrieval of Information Using Special Sources and Data 


0.'.- A 


and Assessment Instruments 


u N A 




D n A 




D I! 4 




D t: A 


' _ 


D A 




D M A 




D M A 




Developed 
N= Modified 
4- Adopted 


i5. Start-up Co»t$: Total expected costs to procure, 
irmtnll and initiate use of the product. 

Estimated materials cost $4-5 p&r 
module, including assessment instruments. 
Other supporting materials up to $10 
(pro-rated) per series. Staff training 
time 2-3 days. 


16. Operating Costs: Projected costs for continuing 
ur^e of product after initial adoption and 
installation H^e.^fecs^ consumable supplies, 
special staff, trainin'j, etc.). 

Not known as yet, but would be equivalent 
to a 3-semester hour college course. 


17. likely Morket: ir^a:^ is the likelu remarket for this prr,duci? Consider the. eize and type of 
th( useryrcrup;^ >nir}ber of po^oiblo cuhotitute (cometitor) products or: the- market; and 
thr- Jikeltf availabi ,itu of fuyxds to purchase product by (for) the produce user prouv* 

Entry professionals with the skills covered in this program will be in demand by those 
employers with the la'-gest (and usually high priority) needs: e,g., development 
programs and projects of educational laboratories, R&D centers, universities, non-profit 
agencies, and industry and business. Certainly federal, state, intermediate, and Tocal 
educational agencies will also provide some employment opportunities, but probably to 
a smaller extent. 
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NCERD Reporfing Form - Developmenfal Producfs 



r 



I. Nam* of Product Functional 
Competence Training Model for 
Development, Dissemination and 
Evaluation Personnel Series 3: 
Communications Skills 



2. loboratory or CanUr 

Far West Laboratory for 
Educational Research and 
Development (FWLERD) 



4. ProbUm: 



description of the eduoatimal problem this product designed to solve. 



3. Report Prapnrotion 

Date prepared January 10. 1 9/ 3 

Reviewed hy 

Laboratory Director 



There is a demand for personnel who are qualified to perform at various professional 
levels -.n the field of educational research, development, dissemination and evaluation, 

JSm™nira7?nfnJTr'"r'°"'-'^''^^'' ^"^^ """^ ^'^ particularly critical, because the 
commum cation of information is a constantly recurring and often very important 

employers that inadequacies in communications skills are frequent and troublesome. 



5. Strategy: ^^e general strategy selected for the solution of the problem above. 

An approach involving job and task analysis, specification of behavioral objectives, 
and formative development of materials and systems was selected as the general strategy 
The design requirements included: a resource-based, functional competence aonroach; 
multi-level programming, modular instructional units, a learner-centered and learner- 
active method, transportability and for long-term cost-effectiveness. 

Communications skills was judged to be high priority in development. The development 
emphasizes teaching communications skills in a functional, task-oriented setting. 



6. Release Date: Approximate dai.i 
vrcdvr.-t -.ns (or r:'lZ boi re^rj 
for vclcasc to >:r.'*; noenau. 

Second Quarter, 1974 



7. Leva) of D^vaiopmanf! Charaat 
level (ov ^.I'OiecU-J. lev^ . 
of <i^v^lopmeKt of pr*cdttc'^ at t\ 
of release. Ciec/! one. 

^^^<^du for critical review and -"or 

prcTaratio\ for Held Teou 
prototuf: nater*n.l:' 

^cad^^* for Field Tecv 

_X Bead:' for publisher modi fioaticn 
J^eaJ^ for general discenCKation/ 
*\\\iSion 



8. Ntxt Agency: 

prcvtcc Dr.- (rr 



United States Office 
of Education. 



10.71.A (D) 
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9. Product D.,crlption: Daaoribe the following; r^^r each deaaription. \ 

•I. Characteristios of the product. .4. Associated products, if arn.. 

• 2. How it. worke. » c c„,^v^, . 

9Pecial condttvone, time, training, I 

• S. What it is intended to do. equipment and/or other requirements I 

for i.ta use. 

1. The Communications Skills series is a ski 11 -oriented, job related oroaram 
g?o"u"s'"?hfs rS"ls"la^o1l57^'' '''''''' f''^'^' b/bo^rlndW^dlTL small 

exe?cises'''° ^^^^P^^^^ion. Much of the wo?k will be presented 'as commutation 

learSnq'nhase "ali .^n IlJPj^'^^"*^^ V" three phases: a preparation phase, an intensive 
learning phase, and an application phase. During the preparation phase a candidate 
for training receives detailed information about the cWtT^as a guidance seL ion 
w e hp 'hP^^fn^nrrf then takes a pretest to determ ne 
^^ntlL^v^ 1 . ^ h^ accepted, he proceeds into the 

0%^' "^l""* P^''^- -I^^ t""^""^^ with a set of learning episodes within 

?nS^ h-^^Tu"^ °" ' 'P''^^^^ ''^^^"^"t self-evaluating checkpoints and counse ?nl 

JarS .HI?! ^'n ^'°^'tl'- ^'"''^^ simUted on-the-job applications oTt ef 

Serf the IJaini^Hi;^'^' aMjcat^n phase, the trainee begins an engineered internsS?p 
Tmc ^'^^'"tu ^f^^l°PS a plan of work that he would like to execute at a particular 
the nlafJnH'.oI nn ^^P^^i^"^ °! the, trainee, the trainee, and his IRM negotiate 

the plan and set up the internship. The engineered internship makes possible the 
!nflI\^-;^°" °I job responsibilities and assignments, establishes performance checkpoint} 
and provides a functional situation in which the trainee applies the skills he has I 
learned, and acquires new ones under realistic conditions." This phase may be 
accomplished concurrently with the learning phase. 

3. The trainee who has completed' this series will have developed the competences to: 

a-, describe the nature of the flow of communication within a DD&E agency; the 
functions and structure of the major kinds of communiques and documentation; 
the use of models , charts, audiovisual media, and techniques of oral 
presentations; and the simple skills used in extending and maintaining 
relationships to establish rapport and facilitate communication. 

b. plan and outline simple communications; develop and conduct briefings and 
demonsirations; present informational materials to groups or organizations; 
write user instructions for completing questionnaires; write simple progress 
reports and memos; write summaries of test data; prepare scripts for filmstrips 
or other audiovisual presentations. 

4. This series is related to and can be linked with the Planning and Design, and 
Information/Data Collection and Organization series. These three series jointly constitute 
a significant cluster which provides foundation skills for the remainder of the program f 
of instruction. 

assiSnina's^lli'hnM.c^' "''^'^ ^° ^°P^ ^^^h this series resulted in 

verbal ?nnimn!L? "'i; T series. Because communication skills include 

verbal communication and personal preservice and inservice D,D&E personnel and 



(continued on page 1 of insert) 



1 A Functional Competence Training Model 
for Development, Dissemination, and 
Evaluation Personnel at Professional and 
Paraprofessional Levels in Education - 
Communication Skills Course 



Product Description: (5 - Continued) 

interaction skills, facilities in which a group of those involved in the course 
can meet together are necessary. 

The series also requires an Instructional "Resource Manager who will coordinate 
facilities, materials, and student interaction. He will also serve in a resource 
and evaluation capacity when needed. 
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to. Product Ut«rti Tkoae individuals or grvupa expected to me the product. 



Continuing education students are expected to use the series. 



11. Product Outcotnott The changee in mar tthaoicTt attitudes, e/fioier-uy^ cts. resulting 
from product use, as supported bu data. Please cite relevant i?'-jpiiort doaumenta. If 
claims for the proSuat are not yet supported by empirical evidence please so indicate. 

The series is presently undergoing formative developnicrtt. At present, no data on its 
effectiveness are available. 



12. Pot«ntigi Educgtionai Cont«(|u«nc«t< Discuss not only the theoretical (i.e. conaeivable) 
implications of your product but also the more probable implications of yoidr product, 
especially over the next decade. 

Manpower supply and economic reality require that development work be accomplished with 
relatively large ratios of professionals. The D,D&E program in general and this series 
in particular is aimed at preparing personnel on the professional level to perform tasks 
at the beginning levels of development, thus freeing the highly trained or more 
experienced developer for work on the more advancdd aspects of D,D&E. This situation 
will be both more efficient and cost-effective. The demonstration of an 
economically justifiable place for the entry professional represents a major challenge 
for the Far West Consortium's design. However, it is hoped that a successful program 
will lay the groundwork for further training and employment. 
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13. Product EUm«nts: 

Hot the elements vhich oonotitute the product. 


14. Origint 

Circle the most 
appropriate letter. 


1 Module 3.1 :Listening and Speaking 


(d) M A 


Assessment Instruments 


DMA 




DMA 


Module 3. 2:Technical Writing: Guidance and Instructional Materials 


(d) m a 


Assessment Instruments 


DMA 




DMA 


Module 3.3:Technical Writing: Work Support Documents 


(d) M A 


Assessment Instruments 


:) M A 




D M A 


Module 3.4:Technical Writing: Formal Publications 


(D)M A 


Assessment Instruments 


D M A 




DMA 




D M A 




DMA j 




D M A 




Developed 
M= Modified 
A^ Adopted 


15. Stort-up Cosfst Total expected costs to procure, 
install atid initiate use of product. 

Estimated materials cost $v-5 per 
module, including assessment instruments. 
Other supporting materials up to $10 

(pro-rated) per series. Staff training 
time 2-3 days. 


16. Opt racing Costs t Projected costs for Cs:^ntinuiKr 
use of product afler initial adoption and 
installation (i'*c. , fees, cone una hie su pp lies, 
special staff, training, etc.)* 

Not known at this time, but might be 
equivalent to a 3-hour semester college 
course. 


17. liic«!/ AAarktt: ITiat is the likely market for this product? Consider the size and type of 
the user gi*oup; mnher of possible substitut^f ( competitor ) products on tic r^arket; and 
the likely availabUity of funds to purchase product by (for) the product user aroup. 

Professionals with the skills covered in this program will be in demand by those 
employers with development programs and projects such as educational laboratories, 
R&D centers, universities, non-profit agencies, and industry and business^ Certainly 
federal, state, intermediate, and local educational agencies will also provide some 
employment opportunities, but probably to a smaller extent. 
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NCERD Reporting Form - Developmental Products 



1. Nome of Product Functional 

Competence Training Model 
£or Development, Dissemination 
and Evaluation Personnel. 



Series 4 : Developmental Engineering 



2. Laboratory or C«nt«r 

Far West Laboratory for 
Educational Research and 
Development (FWLERD) 



3. Report Prtpcratson 

Date prepared Jan. IQ , IQ ?,*^ 

Reviewed by Laboratory 
Director 



4. ProbUm: Oescviption of the educational pvcblem this product designed to solve. 

There is a demand for personnel who are qualified to perform at various professional 
levels in the field of educational research, development, dissemination, evaluation 
and imp lemen t a t i on . 

This need is particularly critical in the area of Developmental Engineering. Through 
literature survey and analysis, time sampling, and task inventories, it was 
determined that Developmental Engineering can be performed by persons with appro- 
priate training at the entry professional level. 



5. Stratogy} 57,^ general strategy selected for the solution of the problem above. 



An approach involving job and task analysis, specification of behavioral objectives, 
and formative development of materials and systems was selected as the general 
strategy. The design requirements included: a resource-based, functional competence 
approach; multi-level programming, modular instructional units, a learner-centered 
and learner-active method, transportability and for long-term cost-effectiveness. 

Developmental Engineering was judged to have a high priority for development. 

The design emphasized teaching the activities involved in this area in a functional 

task-oriented setting. 



6. Release Dat«: Approximate date 
vroduct 'jas (or 'Hit ho) rear.y 
yoleasc to ncrt agency. 

Second quarter 1974 



7. l«v«l of D«v«lopmentt Cnoi^aci — 
Litlj level (or pi'oJect^.S lex^i^ 
of (iav^lcpment of prcdita!: at ti* 
of re lease. Oieck one, 
v:if*a<f^ for critical review and foi* 
prcraratioh for Field 'ics^ 
(V.c. prototype r]atev*fii'<* 

readi^ for Field Test 

heady for publisher modification 

P^easl^ for general dicservlrxLtion/ 

iif fusion 



8. N^xt Agency: 



ri'ciuat wc* 
- jsed 



1*0 



(cr 



10.71. A (D) 



1^1 



1 
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Product OMcription, Doccrite the follauing; nmber each description. 

• I. Characteristics of the product. . 4. Associated products, if any. 



• 2. 

• 3. 



How it worke, 
Vhat it is intended to do. 



Special oondiHone, time, training, 
equipment and/or oth^v requirmente 
for ite use* 



ll.A I °f^^°P™^"'^^l tnginep.ring Series is a skill-oriented, job-related s^t of 
graduate level materials which may be used by individuals or groups as a whole 
or xn separate modules. The materials include frequent feedback and reinforcement 
en^iSe^r experiences in establishing developmental obiecti^es ' 

engineering a sample component of a product, combining components, using test 
^rohZ. " f modification or improvement, coping with special 

problems m development, and locating and employing technical materials, aids 
and resources for development. The modules are accompanied by a battery of 
assessment instruments. "luuery or 

2. The series is implemented in three phases: a preparation phase, an intensive 
innlopr^ ^ V ^" Application phase. In the first, the instructional resource 
manager provides the trainee with information about the series and give3 a pre'est 
Lo determine its usefulness to him. In the second phase the trainee works with 
a set of learning episodes within a module focused" on a specific skill. In the 
^P.'nnr^^^"''' '"'"""f devej.ps. With his *ork supervisor and his instructional 
resource manager, a work plan tv. apply on the job the skills he has learned. 

I'^l^'^^^i"^^ the entry professional level who '.as completed this course 
should have the following .competencies : 

a. he should be familiar with techniques of retrieval and use of materials 
and processes, methods, and subject area content needed for -construction 
of educational products. 

b. given necessary specifications and supervision, he should be able to 

devise and communicate product specifications and development schedules 

c. given necessary supervision and job aids, he should be able to construct 
simple components of educational products, integrate them into larger 
entities, and assist production personnel to develop them according 

to specification. 

d. he should be familiar with the process of review and revision, and the 
establishment of criteria for testing, and final judgment of a product. 

m'^ ^''^n Tli^^ ^' ^^"^^"^ "^'^'^ ""^^ Planning and Design Series, especially 
Module 1.3 Planning for Development". It is also related to the Evaluation Series, 

5. An analysis of the training time has resulted in assignment of 45 credit 
hours of training per semesfcer to this series.* The materials require an 
Instructional Resource Manager who will coordinate faciliti.-;. resources and 
when^"' interaction. He will also serve in a resource an-J -.valuation capacity 



* 8-10 hours per module 
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10, Product Us«rt: Thoec ir^ividualG or grourst expected to uae the product. 

Preservice and inservice DD&E personnel and continuing education students are 
expected to use this series ♦ 



U, Product Outcomoii The cfiangee itt wier behavior, attitudes^ efficiency, etc. resulting 
from' product uee, ao supported bif data * Please cite relevant support documents* If 
clavns for the proSuct are not yet supported by empirical evidence please so indicate^ 

This series is still undergoing fonnative development, therefore data as to 
its effectiveness are not yet available. 



12. Potonttol Educational Contoquoncjit: Discuss not only the theoretical (i.e. conceivable) 
implications of your product but also the more probable implications of your product, 
especially over the next decade. 

Manpower supply and economic reality require that devslopment work be accomplished 
with relatively large ratios of prof essionals • The DD&E program in general and 
this series in particular are aimed at preparing personnel at the entry 
professional level to perform tasks at the beginning levels of development, thus 
freeing the highly trained or more experienced developer for work on fche more 
advanced aspects of DD&E, This distribution of tasks will be both efficient and 
cost-effective. The demonstration of an economically justif iable'place for the 
entry prof essio ia> represents a major challenge for the Far West Conoortium^s 
design, Howev it is hoped that a successful program will lay the groundwork 
for furt:h.?.r trainirig and employment. 



13. Product El«m*nts: 

Liat the elements which constitute the product* 


m I 

M. Origtnt 

Circle the nost 
appropriate letter* 


Module Estabiir^hing Developmental Objectives 


(d)n a 


noo caaiudl L JLlla L L UluSn L S 


f) !< A 


Module 4.2: Engineering a Component and Assessment Instruments 


(d\ M A 


Module 4.3: Integrating Product Components and Assessment Instruments 


(£) iV A 


Module 4.4: Try out and Revision and Assessment Instruments 




Module 4*5: Special Problems in Development and Assessment Instruments 


M A 




D !! A 


• 


M A 




D A 




D !'l A 




D M A 




D /. 




D M A 




l: M A 




IMA 




D= Developed 
r/= !4odified 
w= Adopted 


15. Stort.up Costs: ^^^^^ expected costs to procure, 
install and initiate use cf the product* 
Start--up costs cannot be assessed at this 
time, however, it is anticipated that the 
cost of this series will be roughly equi- 
valent to that of a three--hour university 
course. 


16. Operating Costs: ?rc,iected costs for ccntinuing 
use of producv after initial adoption a>id 
i,Y-otan^^,ic^ (i.e.,fecsy co>isu-^.able supplies^ 
special s'taff, training, eta.)* 

Operating costs are not yet known, 
however, they will probably be 
roughly equivalent to those of a 
three hour university course. 


17. Likely Market: :rnat is the Likely rnarket for this vroduct? Consider the sizr. and tup*? of 
the user^group;^^nur:ber of po'siblc substitute (conpetitor) products on the narket; and 
the likely availabilitu of funds to purchase product by (far) the prodwjz user arout. 

Entry professionals with the skills covered by this series will be in demand by 
those employers with the largest (and usually high priority) needs: eg. , development 
projects of educational laboratories, R&D centers, universities, non-profit agencies, 
and industry and business. Federal, state and local educational agencies will also 
provide some employment opportunities. 



-4- 
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NCERD Reporting Form ~ Developmental Products 



1. Nome of Product Functional 
Competence Training Model for 
Development, Dissemination 
and Evaluation Personnel ♦ 
Series 5: Evaluation 



Loborotory or Center 

Far West Laboratory for 
Educational Research and 
Development (WLERD) 



3. Report Preparation 

Date prepared J ^n > 1Q 



J323 



Reviewed by Laboratory 

4. Problem: Description of the educational problem this product designed to solve. 

There is a demand for personnel qualified to perform at various professional levels 
in the field of educational research, development, dissemination, evaluation, and 
implementation. 

In the area of Evaluation, the need is particularly critical. Through literature 
survey and analysis, time sampling, and task inventories, it was determined that 
evaluation is an activity in the D,D&E process which can be performed by persons with 
appropriate training at the entry-professional level. 



5. Strctegy: j.^^ general strategy selected for the solution of the problem above. ~ 

An approach involving job and task analysis, specification of behavioral objectives 
and formative development of materials and systems was selected as the general 
strategy • The design requirements included: a resource-based, functional competence 
approach; multi-level programming, modular instructional units, a learner -centered 
and learner-active method, transportability and for long-terra cost-effectiveness. 

Evaluation x^as judged to be of high priority in development. The development 
emphasizes teaching in a functional, task-oriented settings. ^ 



6. Releate Dote: Approxinata daie 
vrc^duct v^zs for "ill bcj re2.:* 
f:>r release to k^:-: agsfioy. 

Second quarter - 1974 



7. Level of Development: Cnaract 
Ist:.^ level (or pi-o.'pctf.I hn^e 
of development of prcducz at ti' 
of release, ChecK one. 
X^:rpadi* for criUcal revieo and -''or 
pvcraratioK for Field Test 
protC'tijpc r.ater'' 'il \' 

.-f^^^/ for Field 7e<:z 

Sf^y .^or publisher modificati:)n 
Jiea.y for general disseril'iation/ 
Vffiision 



8. Next Agency: A^eKc^ zr vh 



^0.?1.A (D) 
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Product Descriptions Describe the following; number each deacription. 

• 2. Characteristics of the product, • 4. Associated products, if any. 

• 2. How it works. 9 s. Special conditions, time, training, 

• 3. What it is intended to do. equipnent and/or oth^r requirements 

for vts use. 

1. The Evaluation Series is a skill-orientated, job-related, series consisting of 
modules that permit flexible scheduling by both individuals and small groups. The 
series is largely learner-managed and emphasizes personal interaction, iimnediate and 
frequent feedback, and special student activities. The series will provide instructionjil 
experiences designed to develop specific skills in finding the best way to evaluate if 
a product meets its established objectives. 

2. The series will be implemented in three phases: a preparation phase, an intensive 
learning phase, and an application phase. During the preparation phase, a candidate 
for training receives detailed information about the series and has a guidance 
session with the Instructional Resource Manager (IKM) . He then takes a pretest (a 
simulated task) to determine if he can profit from the training. If he is accepted, he 
proceeds into the intensive learning phase. The trainee then works with a set of 
learning episodes within a module foc*^sed on a specific skill. Frequent self- 
evaluation checkpoints and counseling inform him of his progress. The modules provide 
simulated applications of the target skills. During the application phase, which may 
operate concurrently with the learning phase, the trainee is tested on his competencies 
and then begins an engineered internship. The trainee develops a plan for work that 
he would like to execute at a D,D&E agency. The supervisor of th'e^ trainee, the 
trainee, and his IRM negotiate the plan and set up the internship.. The engineered 
internship makes possible the specification of job responsibilities and assignments, 
establishes performance check points, and provides a functional situation in which the 
trainee applies the skills he has learned, and acquires new ones under realistic 
conditions. 

3. The Evaluation series is intended to provide instructional experiences in {a) the 
logical and methodological basis of evaluation, (B) the kinds of evaluation peculiar to 
DD&E activities and (c) techniques and procedures for coping with practical evaluation 
problems. 

4. This series is related to and can be linked with the Data/Information Collection 
and Analysis series; it will be recommended that they be used concurrently. 

5. An analysis of the training time resulted in the assignment of 8 to 10 
hours of work with each module in the series with fractional credits per module. 



The course requires an Instructional Resource Manager who will coordinate facilities, 
materials and student interaction. He will a] so serve in a resource and evaluation 
capacity when needed. 
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10. Product Uturtt Those individuals or groups expected to use the product. 

Preservice and inservice D,D&E personnel and continuing education students are 
expected to use the course. 



U. Product Outcomoji The changes in user behavior, attitudes, efficiency, etc. resulting 
from product use, ac supported by data. Please cite relevant si4pport documents. ^ If 
claims for the preset are not yet supported by empirical evidence please so indicate 



The series is still in early developmental stages: as a result, no data on its 
effectiveness are yet available. 



■ 

12. Potential Educational Cont«qu6ncet: Discuss not only the theoretical (i.e. comieivable) 
implications of your product but also the more probable implications of your product, 
especially over the next decade. 



Manpower supply and economic reality require that development work be accomplished 
with relatively large ratios of entry-level professionals. The D,D&E program in 
general and this series in particular is aimed at preparing personnel on the entry 
professional level to perform tasks on the lower levels of development, thus freeing 
the highly trained or more experienced developer for work on the more advanced 
aspects of DD&E. This situation will be both more efficient and more cost-effective. 
The demonstration of an economically justifiable place for en try- level professionals 
represents a challenge for the Far West Consortium's design, as employers have not 
traditionally seen a need for specially trained personnel at this level. However, 
it is hoped that a successful program will lay the groundwork for further training 
and employment. 
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13. Product E^omtnH: 

List the eleme>xts -Jhich constitute the product. 


U. Origin: 

Circle the rr,ost 
appropriate letter* 


Module 5*1 The Role of Evaluation in D,D&E 


(U) k A 


As55es^ment Iti53 truis-^nts 


D M A 


Module 5,2 Tests and Measures 


A 


Assessment Instruments 


r iV A 


Module 5,3 Development of Instruments 


g)« . 


.Ai5f?Gssi?]ent Instruments 


.0 M A 


Module 5,4 Field Tests 


(F)L' A 


Assessment Instruments 


D /.' A 


Modnlp S.S Plvfi.lvati'?!! Problems 






D ;V A 




D M 




D :4 




D ;V A 




U t\ 




D M A 




D= Developed 

.^edified 
A'-' ^dotted 


'^5. Stort-up Costii Total expected costs to procure, 
install and initiate use of the product. 

Not know at this time; however, it is 
anticipated that the cost will be roughly 
equivalent to that of a university course. 


16. Operating Costs: Pr jected nosts for ::..nilnuina 
use of r^roduct afti^r initial adoption 2nd 
installation (i.e. , fees, consumable supplies, 
special staff, training, etc.). 

Not known at this time; however it is 
anticipated that the cost will be roughly 
equivalent to that of a university 
course. 


«17. likely Morkett ir^^at is the l^'kely market for this product? Consider the size and t:/pe of 
thd user gr:?up; n:4rnber of possible substitute ^competitor) products on the market; and 
the likely availabiiitv of funds to purchase product by (for) the product ur^er arouf. 

Entry-level professionals with the skills covered in this program will be in great 
demand by those employers with the largest (and usually high priority) needs: e,g., 
development programs and projects of educational laboratories, R&D centers, 
universities, non-profit agencies, and industry and business. Certainly f^ ieral^ 
state, intermediate, and local educational agencies will also provede some employment 
opportunities, bur. probably to a smaller extent. 
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NCERD Reporting Form ~ Developmental Products 



1. Nomt of >rodvcf Functional 

Competence Training Model 
for Development, Dissemination 
& Evaluation Personnel. Series 7 
Dissemination & Marketing 



2. Loborofory or C«nf«r 

Far West Laboratory for 
Educational Research & 
Development (FWLERD) 



3. K«porf Pr«porafion 

Date prepca^ed .T an. 

Reviewed by La boratory 
Director 



4. Probl«m: Description of the educational problem this product designed to solve. 

There is a major demand for personnel who are qualified to perform at various 
professional levels in the field of educational 'research, development, dissemination, 
and evaluation. Surveys of current training programs for developers indicate 

that the output is miniscule in relation to the demand. 

In the area of dissemination and marketing, the need is particularly critical. Through 
literature survey and analysis^ time sampling, and task inventories, it was determined 
that dissemination and marketing are activities in the DD&E process which can be 
performed by persons with appropriate training at the entry professional level. 



5. StroUgy: general strategy selected for the solution of the problem above* 



An approach involving job and task analysis, specification of behavioral objectives, 
and formative development of materials and systems was selected as the general 
strategy. The design requirements Included: a resource-based, functional competence 
approach; multi-level programming, modular instructional units, a learner-centered 
and learner-active method, transportability and for long-term cost-effectiveness. 

Dissemination and marketing were judged to be of high priority in development. The 
development emphasized teaching these activities in a functional, task-oriented setting. 



6. Rolaai* Dot*: Apprcxirnate dan 
Dpcduct i^as (or rill be) vex\ 
for vo least, to ncrt agenoy. 

Second quarter 1974 



7. L«v«l of Dovolopmenh Cnaract 
itnla level (or ;.yo^'eaU,d level 
of development of f rciitcjr at tv 
of release* Chect: one* 
Kjipgdij for critical review and fov 
preraratioK for Field :eCu 
^V.u. protr^tupc rater 

P.eady for Field TeL-t 

P.eady for publisher modification 

P^ea^^/ for general disservl'Zatioa/ 



. usvon 



8. Noxt Agency: 

yvodixcz wa.' 

-'ased fr 
dT'r-lopnent 



10.7!-A (D) 
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Product D^scriptiont 



• I. 

• 2. 

• 3. 



Describe the following; msnher each description. 

• 4* Associated products, if any. 



Characteristics of the product 
Hod it vorke. 

What it is intended to do. 



• £. Special conditions, time, training, 

equipment and/or othf>r requirements 
for its use. 



1. The Dissemination and Marketing Series is a skill-oriented, job-related training 
consisting of instructional modules that permit flexible use by both individuals 

and small groups. The series is largely learner-managed^ and emphasizes inter- 
personal interaction, immediate and frequent feedback and special student activities. 
The series will provide instructional experiences designed to develop specific 
skills in finding the best way to disseminate and market a product that meets its 
established objective. 

2. The series will be implemented in three phases: a preparation phase, an 
intensive learning phase, and an application phase. During the preparation phase, 
a candidate for training receives detailed information about the training and has a 
guidance session with the Instructional Resource Manager (IRM) . He then takes a 
pretest to determine if he/she can profit from the course. If accepted 

he/she proceeds into the intensive learning phase. The trainee then 
works with a set of learning episodes within a module focused on a specific skill. 
Frequent self-evaluation checkpoints and counseling inform him/her of his/her 
progress. The modules provide simulated applications of the target skills. During 
the application phase, which may operate concurrently with the learning phase, 
the trainee begins an Engineered Internship, developing a plan for work that he/she 
would like to execute at a R&D agency. The work supervisor of the trainee, 
the trainee, and his IRM negotiate the plan and set up the internship. The 
Engineered Internship makes possible the specification of job responsibilities and 
assignments, establishes performance checkpoints, and provides a functional setting 
in which the trainee applies the skills learned from the modules and acquires new 
ones on the job. 

3. The Dissemination and Marketing Series is intended to provide instructional 
experiences in the field of public relations and educational marketing. The 
training modules encompass relevant aspects of journalism, public presentation, 
publications, visitor reception, mailing lists, market analysis, copyright, etc. 
The overall intent of the series is to provide familiarization with and under- 
standing of media, graphics, reproduction processes, market research, legal 
constraints, communication techniques, etc., as they apply to educational DD&E. 

4. This series can be linked with the Communication Skills Series. Opportunity 
for application and practice of the competencies taught in this series has been 
designed into all series in the program. 

5. Analysis of the training time needed to cope with this series has resulted 
in the assignment of 8 to 10 hours for each module in the series with fractional 
units of credit per module. The course requires an Instructional Resource 
Manager who will coordinate facilities, materials, and student interaction. He 
will also serve in a resource and evaluation capacity when needed. 
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10. Product Ui«rit Thooe individuals or groups expected to use the product. 

Preservice and inservice DD&E personnel and continuing education students are 
expected to use this series. 



11. Product Oufcom«tt The changes in user behavior, attitudes, efficiency , etc. resulting 
from product use, as supported by^ data. Please cite relevant support documents. If 
clavns for the product are not yet supported by empirical evidence please so indicate. 

The series is still in early developmental stages. As a result, no data on 
its effectiveness are yet available. 



12. Potential Educational Coni«qu«nc«i: Discuss not only the theoretical (i.e. conceivable) 
zmpltcattons of your product but also the more probable implications of your product, 
especially over the' next decade. 

Manpower supply and economic reality require that development work be accomplished 
with relatively large ratios of entry-level professionals. The DD&E program in 
general and this series in particular are aimed at preparing personnel on the entry 
professional level to perform tasks on the lower levels of development, thus 
freeing the highly trained or more experienced developer fcr work on the more 
advanced aspects of DD&E. This situation will be both more efficient and most 
cost-effective. The demonstration of an economically justifiable place for entry- 
level professionals represents a challenge for the Far West Consortium's design, as 
employers have not traditionally seen a need for specially trained personnel at 
this level. However, it is hoped that a successful program will lay the groundwork 
for further training and employment. 




-and_flssessmpnf. Tnc^tnimpntl 



Module 7.2 Working with the Con.um.r .nH Acc... Instruments 



■ tlpdule 7.3 InstaJ Tin^jducati onal Products and A sse .snient Instruments 




15. Start.up Costs: /r^x^? . , 

. Tctal tixpected coats to procure, 

znstall and initiate us(i of the product. 

Not known at this time; however, it is 
anticipated that the cost will be roughly 
equivalent to that of a university course. 



D y, A 



r= Developed 

Modified 
A= Adovttid 



16, Operating Costs: Trojeczcd ac.its for conrinuinn 
use oj proanat after initial adoption and * 
vmtalUtion (i.e., fees, co7:sunable supplies, 
specral staff, t.-aining, etc.). 

Not known at this time; however, it is 
anticipated that the cost will be roughly 
equivalent to that of a university course. 



unxversxt es. non-profit agencies; and industry and business ' ^Jrtainljf ^deral 
state, xn ermedxate. and local educational agencies will also provide some ' 
employment opportunities, but probably to a smaller extent. 
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Appendix I: LONG-RANGE PLANS 

Introduction 

The plans for Consortium activities beyond August 31 , 1974, outlined 
below, are tentative* They are the result of informal discussions and 
tv/o meetings of the Consortium Planning Committee. Plans developed by 
the committee have been adjusted to reflect the decision to lindertake 
'*no further development" in the paraprofessional and advanced professional 
programs. 

Continuation of the Far West Consortium will insure the strengthening, 
refinement, and expansion of Consortium training in the Bay Area, 
the strengthening and refinement of training materials, and the dissemina- 
tion of the training system and training materials throughout the nation. 

Planning for Consortium directions and activities beyond 1974 must 
include consideration of a number of factors based on additional testing 
of the Functional Competence Training System. Test results will provide 
information on the flexibility and adequacy of training to meet changing 
UU&E needs. Consideration must also be given to shifts based on changing 
needs seen by N!E so further program development may reflect training 
requirements current at that time. 
Background 

In outlining our plans for 1974 and beyond we have essentially 
followed the guidelines of the Design Report (1970). However, sucn 
factors as the revised projections of DD&E personnel training requirements 
.by Hopkins (1971), the quest for economy in higher education, and the 
movement towards awarding of credit for off-campus learning, required a 
re-evaluation of our emphasis on pre-service training. 
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Administrators in DD&E agencies have indicated that they are more 
interested in extending and upgrading the DD&E competences of current 
staff than in new hires. In view of the current depressed job market, we 
are projecting that DD&E agancies will be more inclined to solve their 
training needs through the use of continuing education of current employees 
than through employment of graduates of the pre- service programs. Our 
view of personnel requirements in DD&E is best expressed by Baldridge 
and Johnson (1972, p 48): "With the current depressed job market the 
problem is not to find people, of course, but to find and keep the strongest 
people," Currently, the demand for additional personnel in educational 
DD&E is not greater than the supply; but there are still major discrepancies 
between the content of training programs and the actual requirements of 
DD&E positions. We are exploring the possibilities of establishing continuing 
education programs next year in one or two Consortium agencies and in an- 
other agency outside the Far West Consortium geographical region. Information 
obtained on the servicing and training requirements for these continuing 
education programs will provide valuable experience for planning dissemina- 
tion strategies. Thus, the objective of developing flexible continuing 
education programs set forth in the Design Report (1970, p 2,3), has 
taken on a new importance in the light of a changing job market. 

Our plans will be discussed under the headings: (1) Training 
Implementation in the Far West Consortium's Region, (2) Continued Training 
Development; and (3) Dissemination/nistribution. 
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Ir aininq Implementation in the Far West Consortium Region 

Training implementation in the Far West Consortium region would 
continue beyond 1974 in the forms of pre-service and continuing education. 

The Pre-service (MA) program would continue at CSUSF with around 20 
students enrolled in the program. There is a possibility that other 
•institutions of higher learning might offer pre-service programs within the 
region of the Consortium. 

Conti nui ng educati on programs would be offered at participating R&D 
agencies for about 15-25 students. Participants in this program will 
have three options: (1) to acquire course credit for their work through 
extension arrangements, (2) to get certification of competence through the 
use of assessment instruments, and (3) to make informal use of the training 
materials. 

In addition to the above modes, DD&E training might 
be offered in such other forms as institutes and workshops. 

The range and depth of training offered by the Consortium will depend 
on a number of factors: 

1. level of funding for training provided by NIE; 

2. regional or local demand for training at public and private 
educational institutions; 

3. regional or local demand by business and industry for training 
closely allied to educational R&D. 

If, by the mid- 70's, a strong demand for training should emanate 

from state and local educational agencies, they might begin to offer 

inservice training through use of Consortium-developed materials. Continuing 

education might be n.adt: available at cost to nearby private and public 

agencies and business and industrial firms as part of an adult education 
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outreach program, possibly through university extension systems. 
Continued Development of Systems and Materials 

There are two main aspects of continued development: (1) revision 
of the systems based on operational field testjng and; (2) adaptation of 
materials to a variety of delivery forms. 

Revision based on field testing . In section one of this report Wi5 
projected that the Functional Competence Training Program might be introduced, 
in the form of operationai testing, at 10 - 20 institutes of higher learning 
during the first couple of years of dissemination. The monitoring and 
summative evaluation of these programs would require $20,000 - $40,000 a 
year and funds required for revision of the program (around 1977 - 1978) are 
estimated at $80,000 - $120,000. 

Adaptation of materials . To date, primary emphasis has been ou devel- 
oping pre-service programs for entry professional personnel, yet the 
largest potential market may well emerge from the satisfaction of the 
continuing education requirements of DD&E agencies and institutions. 
Therefore, the Consortium proposes to adapt present materials to create 
a comprehensive training system in continuing education. The amount of 
support required for the revision of materials for the continuing education 
model is estimated at $50,000 to $80,000. 

Another extension of the Consortium's effort .would be development of 
the training materials for workshop and institute training. A repackagihg 
of materials in these forms and the development of relevant guides would 
require $40,000 - $60,000. 
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uissciiiination/Liistn'uutiori 

Implementation progress of the Functional Competence Training 
Program could take the following forms: 

Training Consortia . Training consortia implementing the 
Functional Competence Training Program could vary in size from 
very small, consisting of one OD&E agency and academic institution 
. to very large, consisting of sevral developmental agencies and 
academic institutions. We project five to eight sites in the 
country where the latter might be installed. Numerous small 
consortia may be established to satisfy DD&E training requirements 
wherever the need exists. 
2' Preservice programs leading to MA degree . These programs 
can be disseminated to colleges and training institutions 
throughout the country. (We projected 10-20 such applications) 
Some servicing and training of personnel operating these programs 
will be required. 

3- Continuing education programs . These programs will be 
adaptab'ie to a variety of on-site, off-site; accredited, non- 
credited; supervised, or self -study situations. 

4. Instructional materials ( mndul es) in the DD&E compe tence 
areas. By 1974, we will have developed and tested over 30 
modules at the entry professional level, which will be available 
to satisfy specific training requirements of individuals. 

5- Developmental Consortium Model, This model, with its 

supporting systems, brings a full capability to satisfy developmental 
requirements. The model could be used by small projects (such 
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. as RTB Model Training Projects) or large combinations of agencies. 

Dissemination and support of this model would almost certainly 

require federal funding for a number of years. 
By August 31, 1974, a variety of products will have been developed 
and tested. Those products range from the Consortium Developmental Model, 
with its supporting systems and its capability to develop and test 
materials and programs, to the smallest module of a particular functional 
context area. The costs of these products may vary from a few dollars to 
several thousand. The wide variety of Consortium products requires that 
different dissemination/distribution strategies be considered for each. 
We will find some means to distribute the particular product required to 
satisfy the identified need. Our major concern is to find the best way. 

There are several distribution alternatives that should be studied. 
A revolving fiind might be established by the FWL for the distributior;,sale, 
and continuing duplication of materials. Furthermore, the Consortium's 
products can be: (a) sold by the Superintendent of Documents or the National 
Audiovisual Center and promoted by NIE or by members of the Consortium (if 
funded for that task); (b) offered to the educational publishing and knowl- 
edge industries on an RFP or limited-market basis under a five year 
license ., and then released in the public domain; (c) produced and 
marketed by an independent non-profit agency established jointly by Far 
West Consortium member agencies empowered to provide assessment, certifi- 
cation, products and services at cost plus overhead; (d) distributed to 
a network of regional service agencies (such as BOCES, K.ISE, etc.) for 
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reproduction and use in various areas; (e) distributed to university 
extension divisions on a regional basis and then offered on demand; (f) 
distributed to those state education agencies that would commit themselves 
to duplicate the materials and offer the training in adult education and 
pre-service and in-service programs of state colleges; or (g) transferred 
to a national professional organization such as UCEA, AASA, ASCD, AAAS, or. 
AERA which could promote and sell all the components, not just to its own 
membership, but to the other potential markets. 

We also anticipate a number of problems that must be considered in 
our dissemination strategy. Our Instructional System, to be effective, 
requires significant changes in the approach to instruction when compared 
with more conventional programs. Additionally, the instructional effective- 
ness of our products and systems must be demonstrated to potential users. 
Operational models should be available that demonstrate the functioning 
of the Engineered Internship and the Learning Laboratory. Prepared 
briefings and traveling seminars may prove appropriate means of disseminating 
information about the Consortium's products. Finally, test data resulting 
from the summative evaluation of our systeTi and its products will have to be 
published or otherv/ise made available. 



159 



Appendix J: EXPLORATION OF VARIABLES AND ALTERNATIVE CONFIGURATION 

OF THE ENGINEERED INTERNSHIP 

The Engineered Internship provides the trainee with opportunities to apply 
the DD&E competences he/she is acquiring in actual DD&E work situations. The 
Engineered Internship work plan is developed jointly by the trainee, the 
Instructional Resource Manager and the trainee's work supervisor. All three 
meet periodically to plan and evaluate the trainee's progress. The Engineered 
Internship is described in detail in the Instructional Resource Manager's Guide . 
(A copy of this guide was given to members of the site visit team in October.) 

In a letter, dated November 22, 1972, Dr. Susan Klein of the Task 
Force on Researcher Training outlined a set" of que.stions pertaining to the 
Engineered Internship. In response to these questions we will develop by 
April 1, 1973, a design for the exploration of variables and alternative 
configurations of the Engineered Internship and pursue this exploration 
throughout the rest of the project. 

Lines of exploration may include the following: 

1. Use of the Engineered Internship model for academic credit 
versus non-credit for in-service training. 

2. Use of the Engineered Internship with the same Instructional 
Resource Manager and work supervisor for the entire period of instruction 
versus using different Instructional Resource Managers and work supervisors. 

3. Study of the essential characteristics and variables of the 
triad of the Engineered Internship. Effects on trainee competence of the 
patterns of interaction between the trainee and Instructional Resource 
Manager, the trainee and work supervisor, the Instructional Resource Manager 
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and the work supervisor, and among all three will be given careful 
consideration. ( We plan to use multi-variance analysis of variance models 
with quality indicators of the relationship between each pair and, in terms 
of the triad itself, as "design variables" to be related to a battery of 
performance indicators, since deliberate manipulation of these relationship 
variables would be both difficult and impractical.) 

4. The effect of instructional group or team work on trainee 
performance. 

5. A cost analysis of the Engineered Internship. 
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Appeno.x K: A COMPARATIVE ANALYSIS OF TRAINING DEVELOPMENT 
Figure 5a presents a comparison of projected and actual accomplishments 
in the development oF entry-level professional training materials during 
1972. An analysis of the information indicates that in the Planning and 
Design and Communication Skills Series the actual accomplisiin^ents match 
projected accomplishments. In the Information/Data Collection Series, 
one module is in the projected state and four modules are one step behind 
schedule. In the Developmental Engineering Series, three modules are on 
target and two one step behind. In the Evaluation Series,. four of the 
five modules are under revision, however, two of these four require major 
revision and thus are considered "cycled back" for prototype development. 
One module is in prototype development (one step behiad). In the 
Dissemination and Marketing Series, one module is on target and two modules 
are one step behind. The Analysis and Design and Management Series are 
one step behind. 

Figure 5b presents a comparison of projected and actual accomplish- 
ments in the development of paraprofessional training materials . Before 
an analysis is made, it should be noted that developmental activities in 
this program were halted in October, 1972, in compliance with the 
recoiranendations of the site visit team. An analysis of the figure shows 
that three series (Planning and Design, Information/Data Collection, and 
Communication Skills) are one step behind schedule. Developmental 
Engineering is close to target, as some drafts have already been used in 
a course setting. The Evaluation Series is a step and a half behind. 
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Figure 5c refers to the advanced professional program . Instead of 
developing three specific series at this level, we decided early in 1972 
to move toward development of a comprehensive program in R&D leadership. 
A preliminary design was developed and six students are participating in 
a pilot program supported by U.C. Berkeley. 

An overall look at the information presented above indicates that 
the project is behind schedule in some areas. There are several reasons 
that may explain why. First, the scope of the program as presented in 
the design document was developed in anticipation of significantly more 
financial support than was received during the first two years. In 
spite of the reduced funding, we attempted to implement the original 
program scheme. Second, we have involved too many member agencies in 
development too soon, without having adequate lead time to construct 
developmental models, procedures, use guides, etc. Third, in an 
attempt to involve and share resources among many Consortium members, our 
potential to establish a strong in-house monitoring and quality control 
capability was weakened. 

The impact of the conditions described above was manifested in a 
less than desirable state of formative development of systems and materials 
and in a cumulative delay in production. The corrective measures taken 
mid-year of 1972 have moved us in the right direction in coping with 
these deficiencies. Furthermore, in compliance with the recommendations 
of the Fall 1972 site visit team, we have already implemented a scoping 
down of the program. 
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Appendix L: THE EDUCATIONAL INFORMATION CONSULTANT SKILLS IN 
DISSEMINATING EDUCATIONAL INFORMATION 

The Educational Information Consultant Instructional System is the 
product of a recently completed two-year development effort of the Far 
West Laboratory for Educational Research and Development. During the- 
first year of this .project, funded by the U.S. Office of Education's 
Research Training Branch, a 30-hour competence-based training course was 
designed to train persons to perform a role of linking the educational 
practitioner with the output of educational research and development (R&D) 
through dissemination of information about R&D. This person was named the 
Educational Information Consultant (EIC). 

A second year was invested in the further development and testing 
of the EIC instructional system, supported by the U.S. Office of 
Education's National Center for Educational Communication. During this 
phase, three alternative delivery forms of the instructional materials 
(a course form, an instititue form, and a learning team form) were developed 
and validated. These versions incorporated revisions indicated by previous 
testing, and were designed to be complete, self-contained packages, ready 
for use without the Laboratory's supervision. 

The final version of this product is available in two distinct 
packages: a Course/Institute form, designed for use by an instructional 
manager in a classroom setting, and a learning team form, designed to be 
self-administered by a group of at least three trainees and monitored 
through correspondence with an instructional manager at a central 
educational facility. These two forms are identical in content and modular 
sequence. The learning team form differs from the Course/Institute form 
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Retrieval : 



Transformation: 



primarily in the addition of content added to make the materials self- 
instructional ♦ 

The training is based on a model which covers five major processes 
in the EIC role: 

Negotiation: To identify, analyze, assess, and define specifically 

the problem and attendant information need{s) of a client. 
To develop a search strategy and locate, identify, and 
secure R&D information pertinent to the client's problem 
and request. 

To screen, analyze, and/or synthesize and organize the 
results of the search in a form appropriate for delivery 
to the client. 

To display and convey the results of the search to the 
client in a style appropriate for his use in finding a 
solution{s) to the problem. 

To assess the performance of the major EIC processes and 
overall role and the operational effectiveness of the 
setting within the linkage system; reformulate based on 
evaluation^ and make adjustments in processes and functions. 
Knowledge, skills and affective behaviors in each process are presented 
in a sequential, modular configuration. Each one of these major processes 
is the focus of one module. In addition, there is an Introduction Module 
(describing the emerging role and functional contexts of the EIC) and a 
Simulation Module (providing an orientation to the skills involved in each 
process). 



Communication: 



Evaluation: 
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Instructional activities in each module have three phases: (1) 
preparation, (2) learning, and (3) application. Throughout the training, 
participants learn end apply EIC process skills through large- and small- 
group, as well as individual, activities. These include role-playing, 
problem-solving, simulation, and decision-making exercises. 

The EIC materials are one example of materials developed outside of 
our project which attend directly to the DD&E competence defined in our 
training system. The plan is to systematically incorporate these and 
similar type materials into tha system. The EIC materials in the 
learning team form are adapted and incorporated in the Functioral Competence 
Training Program as follows. Three instructional modules will be included: 
Orientation to the Role of the Eductional Inforamtion Consultant, Negotiation/ 
Communication, and Retrieval /Transformation . 

The Orientation Module describes the emerging role and functional 
contexts of the EIC, and provides a simulation of the process of the 
EIC role. The Negotiation/Communication Module attends to skills needed 
to define speci fica'ly an infos' nation problem and to convey the results 
of an "information search. The Retrieval /Transformation Module attends to 
skills needed to secure R&D information and to organize the results of 
that search into an "actionable" format. 

These modules were formed by combining existing modules from the 
EIC training. The learning exercises used in these modules are identical 
to those in the validated Course/Institute operational form of this package.^ 



•5 For a more detailed account see B. Banathy, et al_. The Educational 
Information Consultant : Skills in Disseminating Educational Inforamtion . A 
report on the development and operational validation of three alternative forms 
of a transportable instructional systeiTK Final Report. San Francisco, Calif.: 
Far West Laboratory for Education-! Research find Development. December, 1972. 
U.S. Office of Educational Contract 0EC-0-71-3950(519). 
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However, the content added to bring the materials into the self-instructional 
format of the learning team form is being revised on the basis of the 
findings of the preliminary main field test of that form- This revision 
process is over half-completed and the materials will be ready for 
operational validation in the DD&E system by May 31. The competences re- 
lated to each of these modules have been specified by level. The existing 
assessment instruments will need only minor modifications to bring them 
into the format of the DD&E system^ 

At this time the three modules constitute the ninth series of the 
program. This is a convenient solution for accounting purposes. However, 
we are considering the eventual integration of those modules into the 
Information/Data Series in that competences attended to these modules 
are relevant to those which the Information/Data Series were based. 

A summary of evaluation . The developers adopted the standard that 
for the instructional system to be declared effective, 90% or more of the 
trainees would complete the training, producing an information package 
usable by a client for application to a real-life information problem. 
When assessed for quality, 90% of the trainee-produced information 
packages would be rated at a level of 2.0 (Basic Skill) or above, on 
a 4.0 scale. Additionally, 50% of the packages would be rated at the level 
of 3.0 (Proficient) or 4.0 (High Proficiency). The course form was tested 
at four sites with a total of 67 trainees. The institute form was^ tested 
at eight sites, with 51 trainees divided among 13 teams. 

Ratings of the trainee-produced information packages revealed that of 
both the course and institute participants, 97% did complete the training 
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and achieve the Basic Skill level of competence. Moreover, 74% of the course 
participants and 65% of the institute participants, completed information 
packages that were scored at a "Proficient" or "High Proficiency" skill 
level. 

The trainees' reaction to participation in the training was highly 
favorable. For example, 86% of the course and institute participants 
described the training as "very valuable." Trainees further reported that 
the value of the EIC training compared very favorably with that received 
in other courses and recommended highly that others take the training. 

Since performance standards set for knowledge and skill objectives 
were met at seven of the eight operational field test sites and high 
affective ratings were given the training by both trainees and instructipnal 
managers, the training can be said to be effective. 

These field tests also indicated, for the course and institute forms, 
that the materials are sufficient to enable instructional managers with 
varying backgrounds and experience to effectively administer the training 
and achieve the standards. Thus, these forms of the training package 
may be said to be fully operational and transportable. 

A generally low completion rate for the learning toam form revealed 
that this form cannot yet be said to be operational, and that further 
development of this form is necessary. A revision of this form is now 
under way. 
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ATTACHMENT 0N€ 
Competence Assessment 



This attachment contains a description of the sequence of 
assessment and evaluation steps that will be taken in conjunction 
with the certification of a competence set. Prototype forms of 
the competence assessment instruments for Module 4.2 (Attachment 
Two) are included to illustrate the type of instruments which 
will be used. 

A. DESCRIPTION OF THE ASSESSMENT SEQUENCE 

In Figure 1 we indicate the process by which members of the 
t/iad (Instructional Resource Manager, trainee, and work supervisor) 
may use the assessment instruments for certifying competence. Ihoro 
are two approaches to the certification of competence which may be 
used: the "challenge" method and the instruction and application 
method. 

The following steps outline an approximation of a plan for 
using competence assessment certification instruments: 

Step 1 . For each competence subset relevant to a module there 
is a battery of assessment instruments which members of the triad may 
use in determining trainee competence. The first ste-p is selection 
of a competence subset by members of the triad. This selection is 
usually dictated by the job requirements of the Engineered Internship, 
although a trainee may "challenge" any subset in which he/she feels 
he/she has already attained a degree of proficiency. 



SM^. The trainee completes a Student Self-Rating Scale indicating 
his/her perception of his/her own proficiency in the competence elenents 
included in the selected competence subset. The trainee also completes the 
Job Knowledge Test. 

StefiJ.. The trainee's work supervisor completes the Supervisor's 
Rating Scale which indicates his/her perception of the trainee's level of 
proficiency on the relevant competence subset. Steps 2 and 3 constitute 
a "pre-test". 

Stee_4. Based upon results on the Student Self-Rating Scale, the 
Supervisor's Rating Scale and the Job Knowledge Test, the triad makes 
a decision regarding whether the trainee is to (1) "challenge" the 
competence subset or (2) proceed through- the instruction and application 
phase of the program. If the trainee elects to challenge the module, 
he/she proceeds to Step 5. otherwise trainee proceeds to Step 7. 

StefiJ.. The trainee presents evidence of past performance in the 
competence subset. This may be in the fom of products, reports, exhibits, 
testimony of previous supervisors, etc. If this evidence is sufficient. 
Step 6 follows; if it is insufficient. Step 7 follows. 

Ste£_6. If the trainee has products indicating his/her past 
performance they may be judged by use of the Product Rating Scale. 

Ste£_7- If the trainee has no products which demonstrate past 
experience, or if the products do not meet the minimum criteria as 
judged by the Product Rating Scale (Step 6) he/she proceeds with the 
Simulation Exercise. 

Stee_8. Satisfactory performance in Steps 6 (Product Rating) or 
Step-7 (Simulation Exercise) and 2 Job Knowledge, ad judged by the 
Instructional Resource Manager, results in the trainee being certified 



on the relevant competence elements in the subset (Step 15). If performance 
is not satisfactory on the Job Knowledge Test, the trainee proceeds to Step 9 
(module use). Unsatisfactory performance on the Job Knowledge Test and the 
Simulation Exercise leads to assignment for both module use (Step 9) and 
application experience (Step 11). 

Step 9. The trainee works through the assigned instructional modules 
relevant to the competence subset. 

Step 10 . On completion of the module or modules and application 
experience (Engineered Internship) if both are undertaken, the trainee takes 
the post-module Job Knowledge Test. Failure on this test will result in 
recycle to Step 9. 

Step 11. If the trains^ >ias failed the pre-Simulation Exercise (Step, 7), 
he/she will undertake an application experience (e.g., Engineered Internship) 
in conjunction with his use of the instructional module (Step 9). On com- 
pletion of the application, the trainee may have produced a product. If 
so, he/she proceeds to Step 12 (Product Rating), otherwise to Step 13 
(Post Training Simulation Test). 

Step 12. If the product Is judged as passing the trainee proceeds to 
Step 14. If the product is judged as not passing, he/she proceeds to Step 13. 
(Note: It is reasonable to expect that the trainee may not have been able to 
produce a satisfactory product for reasons beyond his/her control , hence 
provision for simulation testing is provided). 

Step 13. The trainee performs a Simulation Exercise. If he/she fails, 
trainee recycles to Step 11 for further application experience. If he/she 
passes, the trainee moves to Step 14. 

Stee_14. The certification of appropriate levels of competence on the 
relevant competence elements is made by the Instructional Resource Manager 



based on a synthesis of results from all assessment instruments and con- 
sideration of: the Student Log (a log book in which the trainee regularly 
notes his learning and application activities); the work supervisor's 
record of student activities (notes on his/her observations of the trainee 
application experiences); and the Instructional Resource Manager's notes 
on the trainee's activities and progress and the transactions of the triad 
At this point, the trainee and the work supervisor complete the post- 
training rating forms. 

Step 15 > The trainee receives competency certification. 
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B. ASSESSMENT INSTRUMENTS RELEVANT TO MODULE 4.2 "ENGINEERING 
A COMPONENT" (PROTOTYPE FORKS) 

The instruments which comprise the Competence Assessment battery 
are described in the body of the report on page 24. The enclosed 
items are provided as examples of the measurement instruments which 
will make up the Competence Assessment battery for Module 4.2. 
Various forms are being developed before conducting prototype tests. 
Contents 

1. Trainee's Self-Rating Scale. Two different forms, C-1 , 
Style A, and C-1, Style B, of the Trainee's Self-Rating Scale are 
presented. 

2., Supervisor's Rating Scale . One form, C-2, of this instrument 
is presented here. 

3. Job Knowledge Test . One form, C-3, of the Job Knowledge Test 
is presented. Other forms under development include multiple-choice 
and true-false types. We are aware of the difficulty of obtaining 
reliable results with true-false items, but may attempt to determine 
the usefulness of this type of test as a preliminary screening device 
because of its ease of administration. 

4. Simulation Exercise. Form C-4 is an example of a simulation 
exercise which requires the trainee to demonstrate his/her competence 
through application in a simulated work situation. The exercise 
presented here should be completed in four to five hours. A grading 
guide for this exercise is currently under development. 

5- Product Rating Scale . Two forms, C-4, Style A and C-5,5tyle B, 
of this instrument are presented. These scales parmit rating of student 
products. Additional forms are being developed for preliminary testing. 



TRAINEE'S SELF-RATING SCALE Distribution: TRAINEE 

FORM C-1 STYLE A IRM 
Prototype Form - SUPERVISOR 

FWL 

TO BE COMPLETED BY TRAINEE 

TRAINEE'S SELF-PJ\TING SCALE Date 



MODULE 4.2 ENGINEERING A COMPONENT Trainee 



Supervisor_ 



1. Have you had one or more courses in developing educational products? 
•Yes No If yes, describe: 

2. Are you familiar with the processes involved in developing educational 
products? 



Yes No If yes, describe: 

I in specifying t\ 

' program or product)? 



3. How much experience have you had in specifying the purpose of a given 
component (a section or part of an instructional 

Considerable Some Little None 

Describe: 

4. How much experience have you had in specifying performance outcomes or 
objectives (telling what the learner will be able to do at the end of his 
involvement with the component) for a component? 

Considerable Some Little None 

Describe: 

5. How much experience have you had in specifying constraints and resources 
(laying out tasks, working out time, money, and personnel schedules, etc.) 
for a component? 

Considerable Some Little None 

Describe: 

6. How much experience have you had in selecting instructional content (defining 
what content should be provided to the student so that he can achieve the 
performance outcomes) for a component? 

Considerable Some Little None 



Describe: 



Trainee Self-Rating 
Module 4,2 page 2 



7, How much experience have you had in organizing and sequencing instructional 
content of a component in order to facilitate student learning? 

Considerable Some Little None 

Describe: 

8, How much experience have you had in selecting methods and media for presenting 
. instructional content which will facilitate student learning? 

Considerable Some Little None 

Describe: 

9, How much experience have you had In preparing tests and measures to assess 
student performance? 

Considerable Some Little None 

Describe: 

10, How much experience have you had in evaluating the effectiveness of a component 
or product in meeting performance outcomes? 

Considerable Some Little None 

Describe: 

11, How much experience have you had in modifying a component or product to improve 
its effectiveness in meeting performance outcomes? 

Considerablrif Some Little None 

Describe: ^ 



TRAINEE SELF-RATING SCALE 
FORM C-1 STYLE B 

Prototype Form 

TO BE COMPLETED BY TRAINEE 



Distribution: TRAINEE 
IRM 

SUPERVISOR 
FWL 



TRAINEE'S SELF-RATING 

MODULE 4.2 "Engineering a Component" 



Date 



Trainee 



Supervisor 



The list on the next page contains statements of competences involved in 
engineering a component of an instructional product. We want to determine how 
proficient you think you ar« in each of these competences at this time. Read 
each competence statement and check the column at the right that corresponds 
most closely to your estimate of your ability. Seven degrees of proficiency 
are given. These are listed, with interpretations, below. 





Degree of proficiency 


Interpretation 


1. 


No experience 


I have had no experience with this task. 




Read \t observed only 


I have read about or *:»een this task performed, 
but really don't understand it very well. 


3. 


Understand purpose 


I have studied this task or seen it performed 
enough to understand its purpose or function, 
but I haven't ever tried to complete it. 


4. 


Some hand-on-experience 


I have had enough hands-on experience to be 
able to perform it with close supervision or with 
detailed step-by-step instructions. 


5. 


Need only general super- 
vision or instruction 


I have enough experience in performing this 

task to be able to do it if given enough time 

and some general supervision or general instructions. 


6- 


Can perform satisfactorily 
alone 


I can perform this task quite satisfactorily 
without supervision or job aids. 


7. 


Highly proficient 


I have extensive experience with this task, and 
can perform it quickly, efficiently, and do a 
top quality job. 



If you are given a description of the instructional system 
showing the relationship of the component (section or part of the product) 
in question to the instructional system, and given appropriate instructions, 
job aids, and supervision, how well do you think you can perform the tasks 
listed on the following page? 
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TRAINEE'S SELF-RATING 



Self -Rating 



c 
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MODULE 4.2 "Engineering a Component 



II 




p. 2 . 



Tasks 



1. Specify the purpose of a giy^n component of the. 
program (this includes a statement of the problem 
*»hich gives a diagnosis of need and the proposed gener 
ral solutions" to the nee<l). 



2. Specify the performance outcomes or objectives (tell 
what the learner will be able to do at the end of his 
involvement with the component). 



3. Specify constraints and resources (lay out tasks, 
time money and personnel schedules, which spell out 
how we expend our resources and for what). 

4. Select instructional content (define what content 
should be provided to the student so that he can 
achieve the perfomance outcomes). 



5. Organize and sequence instructional content of the 
component to facilitate student learning. 



6. Select methods and media for presenting instructional 
content which will facilitate student learning. 



7. Prepare tests and measures to assess student performance. 



8. Evaluate effectiveness of component or product in meeting 
performance outcomes. 



9. Modify the component or product to improve its effective 
ness in meeting performance outcomes. 



ERIC 



SUPERVISOR'S RATING SCALE 
FORM C-2 
Prototype Form 



Distribution: SUPERVISOR 
IRM 
FWL 



TO BE COMPLETED BY SUPERVISOR 



SUPERVISOR'S RATING SCALE 

MODULE 4.2 'Engineering a Component" 



Date 



Trainee 



Supervisor^ 



The list on the next page contains statements of competences involved in 
engineering a component of an instructional proGuct. We want to determine how 
proficient you think the above trainee is in each of these competences at this 
time. Read each competence statement and check the column at the right that 
corresponds most closely to your estimate of his or her ability. Seven degrees 
of proficiency are given. These are listed, with interpretations, below. 



; Degree of proficiency 
'1 . No experience 

2. Read or observed only 

3. Understand purpose 

4. Some hand-on-experience 



5. Need only general super- 
vision or instruction 



6. Can Perform satisfactorily 
alone 

7. Highly proficient 



Interpretation 
He haslibt had experience with this task. 

He has read about or seen this task performed, 
but really doesn't understand it very well. 

He has studied this task or seen it performed 
enough to understand its purpose or function, 
but hasn't ever tried to complete it. 

He has had enough hands-on experience to be 
able to perform it with close supervision or 
detailed step-by-step instructions. 

He has enough experience in performing this task 
to be able to do it if given enough time and 
some general supervision or general instructions. 

He can perfom this task quite satisfactorily 
without supervision or job aids. 

He has extensive experience with this task, and 
can perform it quickly, efficiently, and do a 
top quality job. 



If the trainee is given a description of the Instructional system 
showing the relationship of the componerrt (sectfon oV^'part of the product) in 
question to the instructional system, and given appropriate instructions, job 
aids, and supervision, how well do you think he or she can perform the tasks 
listed on t he following page? 



SUPERVISOR'S RATING SCALE 
MODULE 4.2 "Engineering a Component" 

p. 2 



T asks 

1. Specify the purpose of a giy^n component of the 
program (this includes a statement of the problem 
*hich gives a diagnosis of need and the proposed gene 
ral solutions' to the need). 



2. Specify the performance outcomes or objectives (tell 
what the learner will be able to do at the end of nis 
involvement with the component). 



3. Specify constraints and resources (lay out tasks, 
time money and personnel schedules, which spell out 
how we expend our resources and for what). 

4. Select instructional content (de^'ine what content 
should be provided to the student so that he can 
achieve the performance outcomes). 



5. Oraanize and sequence instructionril content of the 
component to facilitate student lea*-ning. 



6. Select methods and media for presenting instructional 
content which will facilitate student learning. 



7. Prepare tests and measures to assess student 2?^^orn?nce. 



8. Evaluate effectiveness of component or product in meeting 
performance outcomes. 



9. Modify the component or product to improve its effective 
ness in meeting performance outcomes. 



Rating . 
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JOB KNOWLEDGE TEST 
FORM C-3 
Prototype Form 



Distribution: IRM 
FWL 



TO BE COMPLETED BY TRAINEE 



JOB KNOWLEDGE TEST 



Date 



MODULE 4.2 ENGINEERING A COMPONENT 



Trainee 



IRM 



This test has been designed to determine your understanding and general 
knowledge of the tasks related to developing an instructional component 
or product. Your responses will be compared to those of other students 
with experiences and backgrounds similar to yours. 

1. Define and discuss the importance of each of the following basic 
steps of instructional system design: 

a. stating problem or need and general solution 

b. formulating performance outcomes or objectives 

c. aligning constraints and resources 

d. selecting instructional content 

e. sequencing instructional content 

f. determining instructional strategy 

g. conducting summative evaluation of student performance 

h. conducting formative evaluation of component or product 

2. Define and discuss the relationships of the following elements 
used in developing a component: 

a. performance outcomes 

b. pretest 

c. learning experiences 

d. posttest 

3. Define the following terms: 

a. instructional process 

b. instructional product 

c. instructional system 



SIMULATION EXERCISE 
FORM C-4 
Prototype Form 

TO BE PERFORMED BY TRAINEE 



SIMULATION EXERCISE 
MODULE 4.2 ENGINEERING A COMPONENT 

Instructions to the IRM 

We are asking you to judge the instructional product component that the student 

produces. The performance outcome we expect and the explicit criteria by which 

you should judge the product are included in the attached exercise. 

in addition, we ask you to help the student select another product to work on 

If he is dissatisfied with this one. We anticipate that you will be able to apply 

the same criteria we have listed to the component or product the student chooses. 

Instructions to Students 

In this exercise we are asking you to create an instructional product component 
from the information which we will give you. Refer to Episode II of the module 
if you need further guidance. 

We also realize that in this exercise we are presenting you with an artificial 
situation. It would be highly unusual for you to work alone if you were actually 
engineering a component. Instead, you would have the chance to discuss your 
ideas with many people. However this exercise has an important purpose, which 
outweighs the artificiality of the situation. You should have the chance to 
synthesize the concepts and techniques you have learned by applying them to the 
solution of a specific problem. In short, you should have the chance to 
produce an actual piece of instruction. 

The minimum we ask ic that you produce the component described, although you 
can do more if you wish. You may choose your own component, if you so desire. 
If you do choose to work on another component, be sure and discuss it with your 
IRM before you begin work. 

Required Performance Outcome 

Given the following problem statement, and the table of media costs and time 
requirements, you should be able to produce a prototype version of an instruc- 
tional product component from the information given you about (a) performance 
outcomes, (b) instructional content, and (c) constraints. 

Criteria defining an acceptable product; 

1. It must be completed within four to six hours of the starting time. 

2. It must be developed within the budget allocated. 

3. It must include, whether separately or in combination: 



Distribution: IRM 
■ FWL 



a. performance outcomes 



Simulation Exercise 
Module 4.2 
p. 2 



b. pretest 

c. learning experiences 
post- test 

4. It must be accompanied by a table showing time and money devoted to each 
task. 

5. It must be accompanied by a table showing the medium or media, and 
learning structure or structures you used in the component. 

6. It must be a complete component, something that is usable as a means of 
learning. 

Your product will not be evaluated under any other criteria. 
Your product must meet all criteria to be acceptable. 



Problem Statement 

One of the basic problems in preschool education is that of teaching chil- 
dren age 3-5 to discriminate between basic shapes (circle, square, rectangle, 
and triangle) which are found in the world around them. These shapes appear 
repeatedly in objects encountered every day, however many children are not 
only unable to discriminate among them, but cannot identify the various shapes 
by name. Since children enjoy playing, it has been decided to develop an in- 
structional component which will teach children to discriminate among shapes by 
means of blocks or other games* 



Simulation 
Module 4.2 
p. 3 



Exercise 



COMPONENT DESIGN PROJECTS 



Purpose : To engineer a component that will satisfy the requirements of the 
Problem Statement. 



Your assignment is to: 

1. Specify the performance outcomes or objectives (tell what the learner 
will be able to do at the end of his involvement with the component). 

2. Specify constraints and resources (lay out tasks, time, money and personnel 
schedules, etc* which spell out how we expend our resources and 

for what). 

3* Select or develop instructional content (define what content should be 
provided to the student so he can achieve the performance outcomes). 

4. Organize and sequence instructional content of the component to faci- 
student learning. 

5. Select methods and media for presenting instructional content which will 
facilitate student learning. 

6. Prepare tests and measures to assess student performance. 

7. Describe how you would evaluate the effectiveness of the component or 
product in meeting performance outcomes. 

8. Modify the component or product to improve its effectiveness in meeting 
performance outcomes. 



Constraints : . 
User related— 

1. Must be usable by children age 3-5. 

2. Must be usable in a peer and teacher instructional setting. 
Production related— 

1. Only four hours of your time is available for development. 

2. Funds available for development of this component total $400.00. This 
requires you to allocate your funds wisely for development, reproduction, 
and purchase of materials. 

3. fhe estimated number of users of the product will be ten. 



TABLE OF MEDIA COSTS AND TIME 



Mode of 
Presentation 



Number of hours 

of desian time to Cost per 

produce 1 minute of instructional 

instructional time minute 



Cost of 
your time per 
design hour 



1 . TAPE/SLIDE 

2. FILM 

3. COMPUTER ASSISTED 
INSTRUCTION 

4. AUDIO CASSETTE 

5. PROGRAMMED INSTRUC- 
TIONAL BOOKLET 

6. NARRATIVE BOOKLET 



1 

10 

10 
1 

1 

1/2 



7. LIVE INSTRUCTOR 

(LECTURE-DEMONSTRATION) 1 



$100 
$1000 

$750 
$50 

$20 
$20 

$ 2 



$15 

$15 

$15 
$15 

$15 
$15 

$15 



Explanation of Table of Media Costs and Time: 
Mode of Presentation 

Seven modes of presentation are offered for your use. You must select one or a 
combination of several from this list. You may not use any other mode of pre- 
sentation. The following assumptions are of necessity also in effect: 

1. You must assume that for Modes 1, 2, 3, and 4 an outside consultant 
or media specialist must be brought in (at your expense) to perform 
the design work. 

2. Assume that you are capable without any outside assistance of performing 
the design work on Modes 5, 6, and 7. 

Number of hours of design time to produce one minute of instructional time . 

To compute this cost, complete the following steps: 

1. Determine the number of instructional minutes required for the medio 
in the final product. 

■ 2. Multiply the number of instructional minutes by the Cost Per Instructional 
Minute. 

Cost of your time per design hou? ' 

Your design time must be added to the total cost of the development process. 
This means that you must keep a record of the number of hours you spend working 
on your component and multiply the total by the constant rate per hour of $15. 



PRODUCT RATING SCALE Distribution: IRM 

FORM C-5 STYLE A TRAINEE 
Prototype Form 

TO BE COMPLETED BY IRM 

PRODUCT RATING SCALE " Trainee 

MODULE 4.2 ENGINEERING A COMPONENT 

This questionnaire may.be used to evaluate any instructional oroduct 
developed either as the result of taking module 4.-2 or from some prior 
developmental experience. The questionnaire may be used either as a means 
of challenging the module or in conjunction with the end-of-module exercise. 

1. What financial and personnel resources were available to support this, 
development project? 

2. a. Did you produce this product alone or as a member of a team? 

b. If a member of a team, describe your role and responsibilities. 

3. Who was the intended audience? 

4. In general, what steps did you go through in developing this product? 

5. On what group was the product validated? 

6. What were the results of the validation testing; or if you did not 
have the opportunity to validate your product, describe how you would 
go about doing that. 

7. What did you do to insure learning took place, i.e., that students 
met your objectives? 

8. Would you please show me the pre- test, objectives, and post-test? 

9. Why did you select this particular medium and instructional strategy? 
(This response should relate to the response to (Question 3.) 

Vv). if you considered alternative doproachefi or strategies, at what point 

did you consider them and what were they? 
11. How did you decide on this particular sequence of instruction? 



PRODUCT RATING SCALE Distribution: IRM 
FORM C-5 STYLE B TRAINEE 
Prototype Form FWL 

TO BE COMPLETED BY IRM 

PRODUCT RATING SCALE Trainee 

Module 4.2 "Engineering 

a Component" IRM 



Listed below are ?ome characteristics of an instructional product* 
Please rate the product under consideration on each of these characteristics 
by checking the appropriate column. 



Characteristics Yes No 



1* Objectives: 

(a) are measureable 






& /s^ ^ O ^ 

^ 

^ 4 ^ 


(b) are in performance terms 






(c) include criteria 






2. Intended audience is described 






3. Medium is appropriate 






4. Instructional strategy is 
appropriate 






5. Validation data indicate 
measureable gain 






6. Pre-and post-tests are similar 
(ie. same test, equivalent 
forms, etc.) 






7. Visual Materials: 
(a) technical quality 












(b) contribution towards attainment of 
objectives 












(c) contribution to student motivation 












8. Audio Materials: 

(a) technical quality (audio defects, 
clarity of sound, etc.) 












(b) appropriate to target audience 












(c) interest level (not boring) 












9. Printed Materials: 
(a) technical quality 












(b) appropriate to target audience 












(c) interest' level (not boring) 













PRODUCT RATING SCALE 

Module 4.2 "Engineering a Component 

Page 2 



Trainee^ 
IRM 



10. Instructions to User/Student: 
(a) comprehensiveness 






! 






(b) clarity 












(c) appropriateness of level 












11. Organization/Sequence 












12. /Overall Quality 








i 
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ATTACHMENT yy^o 
Introduction 

This attachment describes the content and format of all modules 
to be developed by August, 1974. The general structure of all modules 
is specified in the Module Developer's Guide. Module 4.2 (Attachment 
Two) is presented as an example. The module descriptions given here 
are in draft form. Also, where modules are undergoing major revision 
or redesign, the description of content is summarized as planned at 
this time. 

Wherever appropriate to or required by the learning situation, 
special instructional strategies will be used in the modules. These 
will include simulations and other special exercises, roie-playing, 
aural and visual mediation, small-team interations, problem solving 
exercises, and broad use of related texts and journal articles. 

For each module, a set of competence assessment instruments is 
being developed. At present, the battery of test instruments consists 
of the following devices: 

1. Student Self-rating Scale 

2. Supervisor Rating Scale 

3. Job Knowledge Test 

4. Simulation Exercise 

5. Product Rating Scale 

These instruments are described in detail in the Scope. of Work Report . 
Part Two: Product Development , page 24. 



erJc 



LIST OF DD&E INSTRUCTIONAL MODULES 



0.0 Overview of DD&E 



1-0 
IJ 

K3 

2.0 

2J 



2.2 



Planning and Design 

The Specification of 1 .4 

Expected Outcomes K5 
Consideration of Alternatives 
Planning for Development 1 



Planning for Evaluation 
Planning for 

Dissemination/Marketing 
6 Introduction to Component Design 



Information/Data Collection and Organization 



Orientation to Collecting and 
Organizing DD&E Information 
and Data 

Observing and Interviewing 



2*3 Data Management 

3-0 Communication Skills 

3J listening and Speaking 

3*2 Technical Writing: Guidance 

and Instructional Materials 



4.0 Developmental Engineering 

4.1 Establishing Developmental 

Objectives 
4-2 Engineering a Component 
4.3 Integrating Product Components 

5.0 Evaluation 

5.1 The Role of Evaluation in DD&E 

5.2 Test and Measures 

5.3 Development of Instruments 

6.0 "l ysis and Definition 

6.1 I lem Formulation 

7.0 Dissemination and Marketing 

7.1 Design and Evaluation of 

Dissemination/Marketing 
Model s 

8.0 Management 

8.1 Management of Personnel 



2.4 Data Analysis 

2.5 The Retrieval of Information 

Using Bibliographic Sources 

2.6 The Retrieval of Information 

Using Special Sources 



3.3 ' Technical Writing: 

Work Support Documents 

3.4 Technical Writing: 

Formal Publications 



4.4 Tryout and Revision 

4.5 Special Problems in Development 



5.4 Field Tests 

5.5 Evaluation Problems 



6.2 Problem Analysis 



7.2 Working with the Consumer 

7.3 Installing Educational Products 



8.2 Management of Program Operations 



ERLC 



9.0 The Educational Information Consultant 

9-1 Orientation 973 Retrieval and Transformation 

9.2 Negotiation and Conmuni cation 



MODULE: 0.0 
Overview of DD&E 



DESCRIPTION OF CONtENT 

The Overview module Is designed to provide a general orientation to 
research, development, dissemination and evaluation in education and 
other social science areas. The content of the module will be derived from 
a carefully selected set of case materials describing actual RDD&E programs 
and projects. The student will be introduced to basic terms and concepts 
by: considering the R & D process as a general problem-solving approach to 
educational and social problems; distinguishing the functions of research, 
development, evaluation and dissemination; being introduced to their nested 
character, e.g., the developer draws on a knowledge base provided by basic 
and applied research, and may in practice contribute to the applied research 
and technology knowledge bases. From the beginning of development, and at 
nearly every step along the way, evaluation is required, design and devel- 
opment must be concerned with dissemination and marketing requirements, etc. 
Following this exposure to the nested, interactive, and iterative character 
of RDD&E, the Orientation module will focus successively on each of the five 
context courses: (a) Analysis and Problem Definition, (b) Planning and 
Design, (c) Developmental Engineering, (d) Evaluation and (e) Dissemination/ 
Marketing. This will be accomplished, in part, by a five part case study 
of a single development. 

FORMAT 

The structure of the module will be similar to that of 'Module 4.2 (see 
Appendix Two). Film strips and audio or videotape recordings of visits to 
development agencies and interviews with DD&E personnel, as well as 
examination of the materials and by-products of DD&E activity will be 
provided as adjuncts to the written case materials. The entire course will 
be basically a single, linear program with exercises requiring student 
response and self evaluation Introduced at frequent intervals; however, 
there will be some alternative sets of exercises and protocols (case 
materials) so that the level of challenge to learning can be adjusted to 
the ability, experience and interest of the student. 



Planning and Design 



MODULE: 

The Specification of 
Expected Outcofffco 



OVERVIEW : 

This module concerns the first overall staoGs of planning. The parts of 
a problem statement are defined, and the procedures for generating the 
parts are discussed. The student is asked to recognize missing elements 
in a problem statement: to plan,^collect and synthesize the necessary 
information; and on the Basis of the problem and goal statements, write 
performance outcomes. 



OBJECTIVES : 

1. Review a problem analysis statement, and identify missing critical 
design and development elements. 

a. Identify the Information given and missing in the six areas to be 
covered in a problem analysis statement. 

2. Become oriented to sources and processes of information collection to 
complete a problem statement. 

3. Discriminate between goal statements and performance outcomes. 

4. Based on a given set of information, write a problem statement. 



DESCRIPTION OF CONTENT : 

The module begins with an introduction that first explains the "downward 
spiral" involved in the process of planning in general. It then discusses 
the development of a problem statement in terns of six categories that 
must be applied. There then follows an exairple of a problem statement, 
specifically the proposal for de/eloping a Parent-Child Education Conponent 
of an Early Childhood Education Program. The six parts of a typical problem 
statement are then broken down and described individually-: 

1. Identification of problem or need 

2. Nature and extent of the problem 

3. Reasons for the existence of the problem 
4* Past attempts to solve the problem 

5. Potential solutions, and 

6. Extent of interest in the problem solution. 



With each description of each part of the problem statement, there are 
student exercises which ask the student to go through and describe each 
part of the problem analysis statement for the Parent-Child Education 
Cornponent of an Early Childhood Education Program (the Toy-Lendiro Library') 
This IS done on a form entitled "Analysis of Information in a Prool-^m 
Analysis Statement" that appears after the first student activity in the 
module. Conpletion of this form comprises the first six student activities 
Student Activity #7 is concerned with the use of A Guide to Referenc e Books 

by Constance M. Winchell. Student Activity #8 provides practice in 

locating sources in the library. Students are given a list of frequently 
used sources and are asked to locate five of them in the library and write 
short descriptive paragraphs on each. In addition, they are asked to vi'iit 
their college library and determine the availability of courses of study 
and curriculum guides. Follcwing Student Activity m are shor> descriptions 
of various methods for obtaining information including interviews, 
questionnaires, observation, surveys, and letters of inquiry. Student 
Activity §9 involves reading a piece (included in the module) about the 
Chinatown-North Beach Comnunity English Language Center, and conpleting 
an Analysis of Information Form. Following that activity is an explanation 
of the differences between Goals and Performance Outcomes. The final 
student activity involves reading from two outside sources: A£RA Monograph 
Series o n Curriculum Evaluation . Instructional Objectives , ani Robert Mager's 
Preparing Instructional Objectives. The student is asked to conplete the 
exe rci s es at the end of Mager's book . 



END-OF-MODULE ASSESSMENT TECHNIQUE : 

Students are asked to write a problem analysis statement regarding 
instructional methods and materials for the development of a time sense 
in cHildren. They are provided with a list of references to aid them in 
this task. They are also asked to write a goal statement and performance 
outcomes for time sense instruction. 



COMPETENCES THE MODULE IS DESIGNED TO ENHANCE : L EVEL OF COMPETENCE 

A-1.1 •: Identifying the Problem Area Familiarization 

A-2.1 Annotating Resource Information Familiarization 

A-2.2 Assessing the Problem Arra Familiarization 

A-4.1 Specifying the Desired Outcomes Familiarization 

A-4.2 Writing the Problem Statement Familiarization 



1. Planning and Design 



MODULE: 1.2 

The Consideration of 

Alternatives 



OVERVIEW : 

This module concerns itself with problem solving and the problems that 
arise which may influence the outcomes. In order to avoid mistakes, 
■certain patterns in this operation must fae followed. These patterns 
ensure every avenue is explored so that the developer m^ resolve any 
discrepancies and solve his problems. These patterns will be described, 
labeled and defined, after which the student will assume the role as 
developer and practice his skills on actual problems. 



OBJECTIVES : 

1. Examine solutions to educational problems and determine whether to 
adopt or reject them, and to justify such decisions in tenns of 
compatibility and feasibility. 

2. Adapt or create solutions to educational problems and justify them 
in ten,TS of compatibility and feasibility. 

3. Write a summary statement that includes a recommendation as to whether 
to adopt, adapt, or create a solution of am educational problem and 
the rationale for his choice. 



DESCRIPTION OF CONTENT : 

Student Activity #1 : The student is given a list of circumstances, 
constraints, resources affecting the problem. After solving the problem 
he will be asked to analyze the procedures and sequential thought processes. 

Student Activity #2: Application of Terminclogy-TemB and concepts of 
objectives, restraints, resources and trade-offs are defined and illustrated 
by exatiples. The developer has choices of adoption, adaption and creation. 
Listed are activities used with each alternative: Adoption - "Taba's 
Social Studies b-nits." Adaption - A new edition of a book, television, etc. 
Creation - create your own approach. Mini-course and Perent/Child Toy 
Lending Library. The end result of using these approaches must answer the 
compatibility and feasibility of Wiese experiences. 




urn 



The outside Reading used in the module is: 

Interaction and Systens (SCIS) Teacher's Guide, Rand McNally Co. 
END OF MODULE AS$ESS^ENT TECHNIQUE : 

Students are asked .to develop a science program for blind elementary 
Chi ldren. Stress is on observation and manipulation of materials, i 



skills also are developed. 



COMPETENCES THE MODULE IS DESIG?jED TO ENHANCE: 



LEVEL OF COMPETENCE 



A-2.2 Assessing the Problem Area 
B-2.1 Analyzing -the Feasibility of Solutions 
B-2.2 Analyzing the Compatibility of Solution 



Familiarization 
Familiarization 



with Problsn 



Familiarization 



1. Planning and Design 



MODULE: 1.3 

Planning for Development 



OVERVIEW : 

The purpose of this module is to provide the student with an orientation 
to some of the factors involved in planning an educational research and 
development project. The module will illustrate the basic steps that may 
be taken in planning for development, and will provide a general orientation 
to the PERT and PPBS approaches to project development. 

OBJECTIVES : 

I. Be able to prepare a detailed project description, with a: 

A. statement of purpose 

B. statement of project objective 

C. identification of the activities to be undertaken by target group 

D. identification of tasks to be undertaken by development team 

II. Be able to structure the format of information concerT?ing a detai led 
project de scription in such a way as to reflect the interdependencies 
and interrelationshtjis of the tasks of the development team. 

III. Be able to determine sequencing tasks by: 

A. describing how development team tasks are identified. 

B. describing the four approaches taken in sequencing necessary tasks. 

C. identifying, after examination of a flow chart, 

T. concarrently occurring tasks 

2. activity - preceding tasks 

3. activity - following tasks 

D. designing a simple flow chart from a list of developrent team tasks 

IV. Be able to estimate duration of tasks by: 

A. describing the difference between probabilistic and deterministic 
time elements. 

B. describing the procedures for obtaining probabilistic and 
deterministic time estimates. 



C. defini ng and discussi ng 



1- the earliest time an event can occur 

2. the latest time an event can occur without 

a. delaying another event 

b. delaying total project time 

3. the concept of critical pathw^ 

4. the concept of free time and slack time 

D. describing and discussing how to make time adjustments to reduce 
total project time. 

V. To become competent at scheduling and the allocating of Resources by: 

A. knowing the constraints inherent in all educational research 
and development projects. 

B. being able to schedule an educational research and development 
project if given project tasks, time estimates, and resource 
allocations. 

C. knowing the uays In which a schedule can be readjusted 

1- "if total project time exceeds time available. 
2. if total costs exceed funds available. 

VII. To be able to estimate costs and prepare budgets by: 

A. -discussing and describing how cost estimates are made. 

B. describing the budget's purpose and the categorization of costs. 

C. describing the difference between the "line item" budget and 
PERT and PPBS budget formats. 



DESCRIPTION OF CONTENT : 

In the module, "Planning for Development," the developer is instructed to 
first define a problem, next, to examine possible solutions and decide on 
one of them. 

Qice a parti -ular solution to a problem is decided upon, the next step is 
to complete • project description which entails : 

(1) preparing a statement of purpose , 

(2) specifying the project objectives (including derived perform? 
objectives) , 

(3) identifying the activities to be undertaken by the tcrget group 
to insure successful performance and the meeting of specified 
objectives , and, 

(4) identifying the tasks which must be undertaken by the developiiient 
team to meet the objectives. 



The "Statement of Purpose," it is stated, must define the goal and examine 
the scope of the project as v/ell recognize the limits and constraints 
Important to the projects cor,pletion. Student Activity #1, which follows 
"Preparing a Project Description," is the first of 5 such student activities 
designed to illustrate or elucidate foj' the student the five major subject 
areas of the module, which are "Designing a Planning System" (which contrasts 
the PERT and PPBS planning tecliniques , either of which help to determine 
the exact scope of the project, identify the work to be done, and estimate 
the length and-cost), "Sequencing of Tasks," Cemphasizing when the tasks 
Identified in the project description must be acconplished, enploying PERT 
networking techniques and the construction of a flow chart ) , "Estimating 
the Duration of Tasks," (dealing with the establishment of a time frame, 
the uses of deterministic and probablistic time estimates, and an introductory 
to Cook's 5 time reduction procedures) and "Scheduling and Allocation of 
Resources," Ctranslating the Hevelopment plan into a time table showing the 
calendar date for start and coirpletion, and assigning resources necessary 
to accomplishing the planned activities. 

The sixth and final step, "Estimate of Costs and Preparation of Budget" 
discusses what goes into estimating the amount of money needed to accoirplish 
the goal of the project and such aspects of budget preparation as Direct 
Costs, Indirect Costs, Fixed Costs, and Variable Costs as exempified in a 
sanple "line" budget contrasted with the same information shown in the PERT 
work breakdown structure fon?. and the PPBS format, completing the final 
phase of the "Plenning for Development" module. 

The Readings in the module are; 

Cook, Desnond L. Educational Pro.lect Management . 

Charles E. Merrill Publishing Co. Columbus, Ohio, 1971 

Banathy, Bela H. , Ed. D. Instructional Systems , 

Pearson Publishers. Palo Alto, California, 1968 

Woodgate, H. S. , Planning By Netvork , 

Business Books Limited. London, 1967 

Nuvick, David "Long-Range Planning Through Program Budgeting," 
Business Horizons, 1969 



END-OF-MODULE ASSESSMENT TECHNIQUE : 

Students are asked to solve a problem using the six steps described above, 

COMPETENCES THE MODULE IS DESIGNED TO ENHANCE : LEVEL OF COMPETENCE 

B-3.1 Establishing Developmental Objectives Orientation 
B-3.2 Preparing a Development Plan Orientation 
B-3.3 Determining Developmental Parameters Orientation 



ERIC 
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1 . Planning and Design MODULE : K 4 

Planning For Evaluation 



OVERVIEW : 

This module is concerned with planning for the evaluation of DD&E projects* 
A sharp distinction is made between the "decision maker" and the 
"evaluation specialist" in the evaluation project. The importance of 
keeping these role distinctions clear is stressed in order that the 
evaluation is adequately designed and performed. After the formal 
presentation in the module the student is asked to develop a plan for 
evaluation of a project. 



OBJECTIVES : 

1. The student will list the procedures used in. preparing a plan for 
evaluation . 

2. The student will identify each type of evaluation information/Data 
presented and describe its purpose. 

3. The student win design a schedule of evaluation activities that 
considers resources available and time ronstraints. 

4. The student will list specific methods of quality control as presented 
in the module. 

DESCR IP TION OF CONTENT : 

This moaule begins by reviewing some of the meanings evaluation has for 
people with different points of view. It then gives a specific definition 
for use in DOSE. 

Section I covers Decision Making and the areas requiring decisions in the 
context of evaluation planning. Then there follows a description of the 
roles of decision maker and evaluation specialist together with the 
relationship between client and evaluation agency. 

Sect^'on II discusses guidelines for evaluation studies with emphasis on 
responsibility, reporting, policies, authority, constraints, and alternatives. 

Section III discusses the functions of the evaluation team and the sequence 
in which these functions are generally performed. At the end is a review 
of the formal presentation. 
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Section IV is concerned with stages of evaluation and gives the student 
an opportunity to put into practice the concepts presented in the module 
in a plan for evaluation of a product. 

Readings : 



Stufflebeam, Daniel L. Excerpt from Evaluation as Engliqhtenment 
for Decision Making 

CALIPERS : Planning the Systems Approach to Field Testing 
Educational Products. The Southwest Educational 
Development Laborator)r. Austin, Texas 

•Jones , Wi 1 1 ard An Operational Model, "Problems and Considerations 
in Educational Development. " Rocky Mountain Educational 
Laboratory. 



END-OF-MODULE ASSESSMENT TECHNIQUE : 

Student Activity §9 in the module asks the student to design a plan for 
evaluation of a product. 



COMPETLNCES THE MODULE IS DESIgjSD TO ENHANCE : LEVEL OF COMPETENCE 

C-3. 1 Planning for Evaluation Familiarization 

C-3.2 Specifying Types of Evaluation Infornj^tion/Data Familiarization 

C-3.3 Scheduling the Evaluation Activities Familiarization 

C-3.4 Specifying Methods of Quality Control Familiarization 



1, Planning and Design 



MODULE: 1,5 

Planning For Dissemination 



OVERVIEW : 

Through the use of pertinent background reading, this module introduces 
the student to prelinrinayy planning of a ccmnercial market structure- 
Based on and using that structure, the enphasis of the module is the 
dissemination and marketing of non-standard educational materials. After 
Substantial reading, the student is asked to 1) outline a disseminati on/marketi 
plan and 2) write a rough draft of a legal agreement for manufacturing and 
distribution- 



OBJECTIVES : 

1- Market forecasting - judging size and type of product to be introduced. 

2, Basic-Market Study - assessing cons uner needs , discovering the 
potential for any product of this type, 

3, Product concept - analyzing its specific qualities. 

4, Product objective - considering size, color, price, quantities, and 
other "dimensions." 

5, Evaluation - finding out if the product will perform its planned 
functions C^ngineering). 

6, Consumer reaction - preliminary soundings to prevent wasted effort. 

7, Models or prototypes - why its best to test in a limited market. 

8, -Production - continuing consumer testing to perfect the product and 
eliminate defects. 

DESCRIPTION OF CONTENT : 

The module begins a sinple analysis of marketing and dissemination in the 
introduction with the discussion of the unavailability of new educational 
materials, not limited to the textbook, and problems of marketing and 
dissemination of such new "products" ideas. 



The first background reading delineates an educational product and provides 
a thorough breakdown of the various aspects of marketing consideration: 
the product, the target group, selection of distribution channels. Subsequent 
background readings provide basic information of the educational publishing 
industry; copywriting the production and the legal aspects of a publishing 
ag reement. 

The first student activity calls on ti\e intern to condense this reading 
into a dissemination and marketing plan of his or her own design using 
a fact sheet, a summary, and a review chart included in the module. Tasks 
are to be divided between the "Agency" and "Publisher." Student Activity HZ 
asks the intern to outline the actions necessary to get a publisher/distributor 
commitment for the "American Government Information Unit," and draw up a 
rough legal agreement between a non-profit agency and a distributor. 



END-OF-MODULE ASSESSMENT TECHNIQUE ; 

The Feedback section of the module includes for assessment, detailed 
J references back to each of the two student activities and a breakdown of 

the competencies necessary for dissemination and marketing of products 
via questions about the same. 



COMPETENCES THE MODULE IS DESIGNED TO ENHANCE : LEVEL OF COMPETENCE 

C-4.3 Integrating Evaluation Components Familiarization 
D-3.1 Designing the Marketing Strategy Familiarization 
D-3.2 Designing a Marketing Component ^ Familiarization 
D-3.3 Scheduling the Dissemination Activities Familiarization 
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1 ♦ Planning and Design 



MODULE: 1.6 

Introduction to Component 
Desi gn 



OVEF^IEW: 



This module is based on creating a "frame work" for the examination of 
designs. Its purpose is to interrelate the artistic design with components 
and simple products, rather than systems, simply to provide a clear, 
relatively uncomplicated introduction to design. Producer, distributor 
and user are the" three aspects of design consideration forming the base 
of the text. Part IV is devoted to comprehensive student activities. 



OBJECTIVES : 

1. For each of the producer, distributor, and user design considerations 
discussed in the module, 

a. Briefly describe or define it. 

b. Give an example of how failure to examine it can lead to a bad 
design, and 

c. Give an example of how careful attention to it can lead to a 
good design. 

2. Show how producer, distributor, and user considerations may sometimes 
conflict. 

3.. Use the considerations co identify and explain potential weaknesses 
and strengths of proposed designs. 

4. Given examples to demonstrate how the quality of design work, i.e. 

the attention given to producer, distributor, and U5;er considerations, 
actually varies greatly among personnel and agencies in the R & D world. 

5. • Use the considerations to help you design a component to meet a given 

problem. 



DE SCRIPT IO N OF CONTENT : 

The text is divided into the three basic components of design: producer, 
distributor, and user considerations. Part four is devoted to analyzing 



and applying design conponent ideas* An outline of the module is as follo\vs 

1 ♦ Producer Consi derati ons 
a* Materials 

Cost; target group application; maintenance; durability of product; 
distributi on 
b* Methods 

Should be selected to meet quality, cost, personnel and time 
constraints 

c. State of the Technical Art 

Designs should be created to meet technological capacity 

d. Personnel Requirements 
Consider Personnel available 

Personnel to be hired: cost, training, availability? 

e. Availability of Critical Resources 
(self e^cplanatory) 

f* Durability of the Product 

(duration) 
g» Evaluation 

Results of formative evaluation are used to improve the design 
before it is completed, 
h* Reproducibi lity 

One of a kind or mass produced 

"All-in-the-head" issue, or logical breakdown easily -pplicable 
to similar situations, 
i. Time 

That time available to produce the product 
j. Cost to Produce 

Control issue which effects almost all other considerations; 
materials, methods, personnel, evaluation 

2. Distributor Considerations 

a. Transporting 

Actual means existent for transportation 

b. Installation 

Does St of installation outweigh other product considerations 

c. Perib.iabili ty 
Ideologic and physical 

d. Styling 

Enhances visual quality; adaptability for moving 

e. Interfacing 

Making component fit into already established distribution system 

f. Cost to Distribute 

3. User Considerations 

a. Avai labi lity 

Easily accessible to user 

b. Ease of use 

Clear instructions, simple operating requirements, reliability 

c. Appropriateness to Target Group 

Reading level, prerequisite skills, taste, values, interests 



d. Requirement for .Supervisory Personnel 
Degree of instructor involvement 

e. Maintenance 
Minimal maintenance 

f. Possibility of Misuse 
Avoidance of uncalculated usage 

g. Safety 

All materials and procedures should be safe 

h. Time 

Effectiveness of product vs." time required in its usage 

i. Cost to Use 

PART IV COMPREHENSIVE STUDENT ACTIVITIES 

No. 4 Alternative Designs for the Recruitment and Selection Component 
of a Training Program 

The objective of this act.ivity js to apply the three component 
considerations to the problem and determine their relevance, to the 
design. Through this process, the student is expected to choose from - 
three possible conclusions a correct solution for the cross training- 
program described. 

No. 5 Design for a Two-Way OTnuni cation System between Citizens 
and Public Officials 

Actual student creativity is asked in this activity which calls for a 
component of a system to be designed by the student. The design should 
incorporate terms of the producer, distributor and users considerations 

No. 6 Considerations tn the Design of a Drug Education Program 

After reading the enclosed article, the student is asked to apply 
four aspects of good design as discussed previously in the module to 
the program. To select and describe two aspects of poor design and 
assess how a shortage of fur.ds affected considerations. 

No. 7 Some Design considerations in the Mini course 

The student is asked to list the design consi de rati ons being examined 
in the mini course, and the mode of examination - such as researdi or 
review of literature. 



END-OF-MODULE ASSESSMENT TECHNIQUE : 

The student will complete the four Activities described above. 



COMPETENCES THE MODULE IS DESIGNED TO ENHANCE ; LEVEL OF COMPE -"^NCE 



B-3.4 Designing Component Systems 



Familiarizat m 



2. Information/Data Collection .and Organization 



MODULE 2.1 

Orientation to Collecting 
and Organizing DD&E 
Infonnation & Data 



OVERVIEW; 

This module is intended to orient the student to the series of Information/Data \ 
Collection and Organization. Using a Final Report of our educational pilot study i 
as the contextual framework, the student is introduced to these categories of I 
data collection: the Literat'jre Review, Observation, Self reporting. Representative 
data collection and management techniques are examined through example, discussion 
and brief exercises. 

OBJECTIVES; 



After working through the module, the student should be able to describe key elements 
of the following methods of data collection and management. 

The Literature Review; problems and procedures involved in the 
search;, one method of organizing data. 

Data Collection through Observation; Problems and procedures involved 
in an observation program; content requirements for a reliable 
observation foTni; meaningful presentation of raw data. 

Data Collection thro.ugh Self Reporting; purposes and methods of 
administration of activity records, questionnaires and interviews. 

Linkage with other Modules; 

This module precedes the other modules in the Information/Data Series. 
Description of Content; 

The module consists of four learning episodes and is accompanied by the -final 
report of an educational pilot study. The contents of the learning episodes 
ai^e: 1. The Research Project, an Overview 

2. The Literature Review; Use of Bibliographic Sources 

3. Data Collection through Observation 

4. Self Reporting; Activity Records, Attitude Questionaires , 
Interviews 

The basic resource that will be used for this moduie is: 

Jung, S.M., Lipfi, D & Carter S. Experimental Assessment of an 
Incentive Program to Enhance School Learning 
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A Pilot Study, Final Report to Franklin-HcKinley School District, Palo Alto, 
California: American Institutes for Research, 1971. 

In addition, the student will be asked to examine the following materials 
used in the Pilot Study: 

Teacher interview form 

Student interview form 

Parent questionnaire 

Parent interview fom 

Classroom observation form 

Instructional practices questionnaire 

Parent record form 

The. student will also read a description of the E-Z Sort file system; read a 
discussion of bibliographic resources used in an actual literature review; 
and examine a sample bibliography page. 

E nrichment & Supplementary Activities : 

There are no enrichment and supplementary activities in the module. 
S tudent Assessment Techniques: 

There are no progress checkpoints in this introductory module. 

The end-of -module test will ask the studer^t to list, define, identify, and 
describe information contained in the module materials. 

C ompetence Elements the Module is Designed to Enhance: Level of Competence 
C-3.2 Specifying Types of Evaluation Information/Data Familiarization 
C-5.2 Organizing Information/Data for Analysis Familiarization 



The Readings Used Which are Outside the Module: 

American Psychological Association, Standards for Educational and 
Psychological Tests and Manuals , Washington, D*C», 1966 

Borg, W.R* and Gall, M.D., Educational Research, an Introduction , 2nd ed.. 
New York; David McKnay Co., Inc., 1971 

Levine, S*, and Elsey, A Programnied Introduction to Educational and 
Psychological Measurement , Belmont, California: Brooks Cole Publishing 
Co- 1970. 

The Optional REadings Are: 

Engelhart, M.D*, Methods of Educational Research , Chicago: Rand McNally 
and Co., 1972 

Sjogram, D.D, "Measurement Techniques in Education" Review of Educational 
Research , Vol > 40, 11, April, 1970 

END OF MODULE ASSESSMENT TECHNIQUE : 

The student is asked to select a standardized test that would be suitable for use 
as a pretest and poi.ttest in determining the effectiveness of a specific 
product's use with the intended audience. 

CGMPETENCIES THE MODULE IS- DESIGNED TO ENHANCE LEVEL 

C-4.1 Preparing Evaluation Instruments Familiarization 
C-5.1 Administering Evaluation Instruments Familiarization 
C-0.2 Assessing the Evaluation Instruments Familiarization 



2- Information/Data Collection and Organization MODULE: 2.2 

Observing and Interviewing 



OVERVIEW; 



The purpose of this module is to enable the student to develop competences in 
observing and interviewing that will be useful in development, dissemina'tion, 
and evaluation (D.D&E) projects. These competences include knowledge of a 
variety ov techniques of observing and interviewing and an understanding of 
basic principles pertaining to their use, as well as skill in their application. 

OBJECTIVES; 



1 



2. 



3. 



5. 



7. 



8. 



9. 



Knows the general nature of a variety of observation methods and 
interview techniques and is familiar with ways in which they are 
commonly used in DD&E projects. 

Can explain why specific factual evidence should be the main 
part of recorded observations rather than general descriptive 
statements or judgments or interpretations of the observer. 

Can explain the need for "focusing" or "pinpointing" the behaviour 
or characteristics to be observed or elicited in observations or 
interviews. 



4.; Can explain how needs arid expsctations of the observer can distort 
observations and how these distortions can be minimized. 



Can explain why the phenomenon of selective forgetting makes it 
important to record results of observations or interviews as soon a<; 
possible. 

Can- explain how the personality, dress, body, language, and manner of 
speaking of the interviewer can influence- interview results, and can 
further explain how these distortions can be minimized. 

Can explain why reliability of observation and interview results can 
be improved by recording what was said or done accurately and immediately, 
by obtaining larger samples of behaviors, and by standardizing procedures 
and definitions. 

Can describe examples of why it is important to record a description of 
the setting in which observations and interviews are conducted. 

Can give examples of how different sequences of questions in an interview 
might produce different responses— because of the order of the questions 
rather than their content. 
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10. Can explain why it is necessary for an interviewer to establish raooort 
with an interviewee. >ap, un. 

11. Can explain why "off-the-cuff" comments by an interviewer— especially those 
which reflect evaluation of responses— can distort interview results. 

12. Chooses wording and format that force the observer to record factual evidence 
as distinguished from general descriptions, judgments, and interpretations. 

13. Includes a statement of purpose, definitions and other guidelines that enable 
the observer to know precisely what he is supposed to observe. 

14. Includes standardized procedures , 'category descriptions and other instrurticns 
designed to minimize distortions due to the needs, expectations and personality 
characteristics of the observer. v j 

15. Provides space and clear instructions for making iitmediate records of 
behaviors. 

16. Prepares items that are varied in such a way as to assure that an adequate 
sample of behaviors will be recorded. 

17. When appropriate, includes space and instructions for recording a 
description of the situation in which the observed behavior occurred. 

Given a relatively simple observation schedule with which he is already 
thoroughly familiar can give an explanation on its use t'nat a co-worker agrees 
.IS clear and sufficiently complete to enable him to use the schedule properly. 

18. States the purpose of the observations. 

19. Describes the types of persons to be observed and the situations 
in which they will be" observed. 



20. 



21 



Explains procedures to be followed, such as when to observe 
and for how long, where to record various types of behaviors, 
and how to summarize observations. 

Explains likely pitfalls in using the schedule and how to 
avoid them. 



22. Given an observation schedule and a self- tested manual on its proper use, 
can learn (by studying the manual) how to use the schedule within the 
length of time recommended by its designers, and can make observations and 
record them on the schedule so that there is at least 80 percent agreement 
with the observations recorded by a fully trained observer. 

23. Can apply these interview techniques to DD&E projects. 



END-OF-MODULE ASSESSMENT TECHNIQUE; 

The student will be asked to prepare, administer and evaluate observation 
schedules and interviews. 

COMPETENCES THE MODULE IS DESIGNED TO ENHANCE ; LEVEL OF COMPETENCE 

C-4.1 Preparing Evaluation Instruments Familiarization 

C-5.1 Administering Evaluation Instruments Familiarization 

C-6.2 Assessing the Evaluation Instruments Familiarization 



2. Information/Data Collection and Organization MODULE: 2.3 

Data Management 



OVERVIEW: 

This module is designed to introduce students to various techniques for 
processing, organizing, and displaying data. Students will have a set of 
data obtained from a DD&E project to work with and will practice many of 
these techniques using the data. Upon completion of the module students 
should be able to organize sets of raw data and to prepare clear and 
informative visual presentations of that data. 

OBJECTIVES' : 

1. Be familiar with the kinds of data and levels of measurement 
that are coiranonly used in educational DD&E. 

2. Know general procedures for organizing data. 

3. Given a DD&E problem and a set of data, organize the data in 
a meaningful way. 

4; Knows a variety of ways to display data. 

5. Prepare a clear and informative visual presentation of a set 
of data. 

6. Interpret a visual presentation of a set of data. 

7. Evaluate a visual presentation of a set of data 
DESCRIPTION OF CONTENT 

The module has five Episodes which will -describe the planning process and the 
skills and knowledge needod to accomplish the problem. 

1. Types of Data and Levels of Measurement (frequencies, 
percents, proportions and ratios are data) (nominal^ 
ordinal ,interval and ratio are measurements) 

2. Methods for Processing and Organizing Data (assigning 
identification numbers, coding and tallying; also 
computer facilities and key punch cards.) 

3. Methods of Presenting Data: Frequency Distributions 
(how to construct frequency distributions; the difference 
between relative and cumulative frequency distributions) 



I 

4. Methods of Presenting Data: Graphical Displays 
(graphic frequency distribution, trbular frequency 
distributions, polygons, histograms, scatter diagrams, 
and interpret and evaluate displays.) 

5. Methods of Present Data: Pictorial Displays 
(pie cfiarts and symbol charts) 

6. Summary and Review 
The Readings are: 



Clarke, Robert, et.al. Statistical Reasoning and Procedures . 
Columbus: Charles E. Merrill Books, Inc., 1965 ■ 

Guilford, J. P. Fundamental Statistics for Psychology and 
Education . New York: McGraw-Hill Book Co., 1969 

Sawin, Enoch I- Evaluation and the Work of the T eachgK 
Belmont: Wadsworth Publishing Co., 1969 

Spence, Janet T. et.al. Elementary Statistics. New York: 
Appleton-Century-Crofts, 1968 

Tyler, Leona. Tests and Measurements . Englewood Cliffs: 
Prentice-Hall , Inc., 1953 

Readings For Data Management , Far West Laboratory for 
Educational Research and Development, 1971. 
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END-OF MODULE ASSESSMENT TECHNIQUE : 

The student is to process and display the data collected from the previous 
module. O bserving and Interviewing . The student should process and organize 
it in a way that seems most appropriate to him or her. 

COMPETENCES THE MODULE IS DESIGNED tO ENHANCE; LEVEL- OF COMPETENCE 

C-5.2 Organizing Information/Data for Analysis Low Prof. 

C^7.1 Displaying Evaluation Information/Data Low Prof. 

C-7.2 Summarizing the Evaluation Information/Data Analysis Low Prof. 

D-6.2 Analyzing and Displaying Marketing Information Data Low Prof. 
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2. Information/Data iColl ectl on and Organization MODULE: 2.4 

Data Analysis 



OVERVIEW: 



This module is designed to be a survey unit on techniques of data analysis 

e II on Vo&f^''' nT L° development.. disseminat Sn. d' 

!SSc"k 1^2" }^ P^^^""^^ ^ 9^"e^3l familiarity with cotton 

Stat shcal teinis aiid concepts. In addition, it will provide opporZiSies 
to use- various eleitiontary statistical procedures and techniques The 
intended purpose of the module is to enable the studeni! after completion 

Sls*c sSi -ica? applications -of refe?enc s to 

JfcLniH ; 1 problems that may arise in educational D,D&E. In other words 
u'sef °"|L^'!L.*f' 5° ^^^^^°P ^" appreciation of statistics and Jheir ' 

uses. This module clearly does not attempt to teach the student the stati<;tiral 

ad'; L3\i:t?:^ical^^'Lln.?^^°"?/^"'^ °' research^'^dSS't^lfln^t'rttiJn" 
chnniH u techniques will be necessary to achieve this proficiency 

JteJ2?I?'thrr.'^.nt'''°r^' student. with sufficient information^ to cJrrebtly 
interpret the results of more sophisticated analysis. 

OBJECTIVES: 

I5n/nl^°l!'"?.°?^^''?y" ^''^ "^^^^"^^ ^" of specific behaviors which the 

student should be able to perform at the conclusion of this module. Yo5 will 
notice that these specific behaviors are listed in terns of proces es ?a?her 
than topics. That is, rather than list all the objectives relating to leans or 

as? a??s;Sr^n*^L?'^ one objective is concerned wit unde?stanS ' 
basic statistical concepts (including means and standard deviations)- anot.W 
IS concerned with .computation of basic statistical measuref ?nclud ng mLn^ 
and standard deviations), and so forth. Thus, the content o- topics that each 
ODjective relates to are often the same for the different processes sta?ed?n 

a 1 s fobiectives'°' IT'l'^ "Tm" ^''^ arHeauS^t "in 

ail SIX Objectives. So, while the list may at first glance seem forebodina 

1. UNDERSTAND BASIC STATISTICAL CONCEPTS. 

a. Explain the meaning of the following terms. 

1) arithmetic mean 8) normal distribution 

il ""^dian 9) probability 

il "«de 10) random sample 

c ... sampli ng error 

5 standard deviation 12) tests of significance 

7) no?ms coefficient of correlation 

b. Given an example of the tenr.s in one of the following pairs 
terns ^ explaining the difference betv/een the two 



1) descriptive and inferential statistics 

2) "central tendency" and "variability" in score distributions 

3) raw scores and derived scores 

4) percent and percentile 

5) population and sample 

6) significance level and confidence interval 

7) research hypothesis and null hypothesis 

8) Type 1 and Type 11 error 

9) correlation and causality 

10) parametric and non-parametric statistics ■ 

2. CARRY OUT COMMON STATISTICAL OPERATIONS 

a. Compute the following statistical measures. 

1) common measures of central tendency (mean, median, mode) 

, 2) common measures of variability (range, standard deviation) 

3) derived scores (standard scores, percentiles) 

4) level of significance 

5j tests of significant differences (v^) 
' coefficients of correlation ( p ) 

3. INTERPRET THE RESULTS OF ELEMENTARY STATISTICAL ANALYSES. 

a. Given data containing the following statistical measures, 
write or state correct conclusions based on the data. 

1) common measures of central tendency (mean, median, mode) 

2) common measures of variability (range, standard deviation) 

3) derived scores (standard scores, percentiles) 

4) level of significance and confidence intervals 

5) tests of significant differences (t,x^ ) 

6) coefficients of correlation (r p ) 

4. APPLY APPROPRIATE STATISTICAL MEASURES AND TECHNIQUES TO VARIOUS 
KINDS OF DATA. 

a. Given a particular problem and a set of data, specify appropriate 

1) measures of central tendency and variability, 

2) measures of correlation, and 

3) tests of significance of differences. 

5. RECOGNIZE INAPPROPRIATE APPLICATIONS AND INTERPRETATIONS OF STATISTICS. 

a. point out fallacious uses of statistics involving the above 
concepts and procedures and give sound reasons as to why they 
are fallacious. 

DESCRIPTION OF CONTENT; 

The module consists of ten learning episodes. The contents of these learning 
episodes are: 

1. An Introduction to Data Analysis 



2. "Averages": Measures of the Center of Score Distributions 

3. Measures of the Variability of Scores 

4. Derived Scores 

5. Score Distributions and Probabilities 

6. Drawing Conclusions based on a Sample: An introduction to 

Inferential Statistics 

7. Forming and Testing Hypotheses ai)OUt Samples 

8. Testing Hypotheses about Data^x? 

9. Exploring Relations Between Groups of Data: Correlation 
10. Summary and Review 

The basic resources that you will use in this module are the following three 
books. 

Elzey, Freeman. A FIRST READER IN STATISTICS. Belmont: 
Wadsworth Publishing Company, Inc., 1957. 

Huff, Darrell. HOW TO LIE WITH STATISTICS. New York: 
W.W. Norton » Inc., 1954 

Spence, Janet T., et al . ELEMENTARY STATISTICS. 
New York: Appleton-Century-Crofts, 1968. 

Other references are also made to the following books— either additional textual 
references, or enrichment or supplementary references. 

Clarke, Robert, et al. STATISTICAL REASONING AND PROCEDURES. 
Columbus: Charles E. Merrill Books, Inc., 1965. 

Downie, Norville M. TYPES OF TEST SCORES. Boston. 
Houghton-Miff}in Company, 1968. 

Gorow, Frank. STATISTICAL MEASURES: a PROGRAMMED TE.XT. 
San Francisco: Chandler Publishing Company, 1962. 

Levine, Samuel & Elzey, Freeman. A PROGRAMMED INTRODUCTION TO 
RESEARCH. Belmont: Wadsworth Publishing Company, Inc. , 1968. 

McCol lough. Celeste & Van Atta, Loche. STATISTICAL CONCEPTS. 
New York: McGraw-Hill Book Company, Inc., 1963. 

Tyler, Leona E. TESTS AND MEASUREMENTS. Englewood Cliffs: 
Prentice-Hall , Inc., 1963. 

Weinberg, George & Schumacher, John. STATISTICS AN INTUITIVE 
APPROACH. Belmont: Wadsworth Publishing Company, Inc., 1962. 

In addition, several activities will involve using the laboratory materials. 
These materials include the following: 

2 Hexstats 

2 decks of cards 

2 distance scales 

2 wooden incline planes 

2 wooden disks 
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the students about the location of these lab 
materials and the hours when they will be available. 

END OF MODULE ASSESSMENT TECHNIQUE 

The student's competence in these areas will be assessed by means of a 

performance test The student will be given a series of problems or situations that 

might be expected to occur in the field of educational DD&E. and he will be 

tt f.lf °' '-K^^' and knowledge to these incidents. Uems comprising 

arranged into a series of hierarchial sections, so that items 
n the higher-level sections will include the knowledge and skills tested in^the 
lower-level sections. Students will begin with the higher-level sections; if 
JhfmnSuTf tLt '^nlh ^".t^'os^ sections correctly, they will have completed 

fnr?h nnJfi ;h O^J^'^wise. they will go on to the next lower-level section, and 
matin.? inJ-J-f f'^" determined the level at which they have mastered the 
h^cfn learning activities may be prescribed by the instructor, 

based on the student's performance on the various sections of the test. 

COMPETENCES THE MODULE IS DESIGNED TO FNHAMrP. LEVEL OF COMPETENCE 

C-5.3 Analyzing Evaluation/Data inwPrnf 
D-6.2 Analyzing and Displaying Marketing Information/Data [qw Prof! 



2. Information/Data Conection and Organization MODULE 2-5 

The Retrieval of Information 
Using BibTiographical SouTces 



OVERVIEW: 

The purpose of this module is to enable the student, to develop competences in 
conducting searches for information of kinds likely to be needed in development, 
dissemination and evaluation (D,D&E) projects using a variety of bibliographical 
sources and other guides for locating information- The competences include 
familiarity with the location and contents of sources and aids, 'comprehension 
of principles underlining procedures for their use, and skills for making 
searches. 

OBJECTIVES: 

1- Is familiar with bibliographic sources and other guides to 
information such as dictionaries, encyclopedias, yearbooks 
and directories, and is able to locate them in a library and 
use them. 

a. Knows tha names of at least two general references that 
contain a detailed listing of bibliographical source.-^ and 
other aids for locating information and 

(1) can find them in a library, and 

(2) can use them to identify sources needed for a 
particular search problem. 

Example: DocumGntation in Education , by BurU-. 

b. Can locate any of the following in the appropriate card 
catalog of a librar . 

1) Author-title entries for books 

2) Subject entries for books 

(3) Document entries 

(4) Pamphlet entries 

(5) Test entries 

c. Can use each of the following to look up specific entries: 

(1) Education Index 

(2) Psychological Abstracts 

(3) Encyclopedia of Educational Research 

(4) International Encyclopedia of the Social Sciences 

(5) " Mental Measurements Yearbook. O.K. Buros 

(6) Dictionary of Education . Carter V. Good 

(7) Statistical Abstract of the United States 

(8) Digest of Educational Statistics . 1970, U.S. Office of Education 



^(9) Education Directory . U.S. Ovflce of Education 

(10) Leaders in Education . Jacques Cattell Press 

(11) National Register of £ d-, ;cational Researcher s. Phi- 
Delta Kappa ■ 

(12) Directory of Educational I nformation Centers . U.S. 
Office of Education " 

(13) Dissertation Abstracts 

(14) Bibliographic Index 

(15) Book Review Digest 

(16) Cumulative Book Index 

,(17) American Universities and Colleges 

(lb) Nev. York Times Index 

(19) Educators Worl? 

2. Can locate and use special equipment and devices typically found in Vibraries 

a. Can find each of the following in a library and use it 
properly: 

(1) Microfilms and microfilm readers 

(2) Microfiche cards and readers 

(3) Photocopy Machines 

3. Comprehends ways in which subjective characteristics of the searcher 
can bias the procedures and outcomes of a bibliographical search, and knows 
ways in which such sources of bias can be minimized. 



a. Can explain how a search can be influenced or distorted by: 

(1) the particular concepts and terminology the searcher 
is familiar with antl prefers. 

(2) The particular bibliographical sources and aids that 
the searcher is familiar with and prefers. 

(3) habits or preferences of the searcher for organizing 
concepts and categories. 

(4) the personal values of the searcher that Influence 

his perception of the relative importance of particular 
documents after they are retrieved. 

for each of 3a (1) through 3a (4) above, can give at least 
one example of how a particular bibliographical aid or search 
procedure can be used to minimize the effects of the particular 
source of bias. 



4. Can conduct a survey of literature related to a given topic or problem 
and organize the results. 

a. Prepares or refines the staterrsent o^' purpose of the survey 
through careful analysis of the; protlem, and, if feasible., 
through interaction with the p6rr>r-n who requested the 
survey. 
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b. Formulates descriptors or keywords to guide the search 

c. Prepares a detailed search strategy. 

d. Retrieves bibliographic entries and documents, and prepares 
bibliographic cards according to a standard form, ^^'^^'''^^^ 

e. Scans, evaluates and screens retrieved documents 

;!;S^n!!c^5^!r^*' annotated bibliography of documents 
that passed the screening, 

g. Classifies, indexes and organizes bibliography cards and 

rSri^nfin-'^'^i'"''^*^^*' alphabetically, by type of document, 
chronologically, or as requested by the person who asked for 



DESCRIPTION OF CONTENT : 




The titles of the learning episodes are: 

J- Introduction and Library Tour 

2. Principles and Procedures for Library Searches 

■ Lib?arJ^ Bibliographical Sources and Other -Library Aids in Another 

^- Searches for Specific Items of Information 

Conducting Actual Searches on Practical Problems in DDSE Projects 

The student will have much of the responsibility for his own learnina ' The 
learning activities are designed so that the instructor can serve Is a guide 
'™ ^"^""^""9' the most part! student 11 ' 

ihe ;ns{r2a:?.*'' ''''' ^" ''''''''' ''^^'^'^ "y frol 

The references in order of appearance in the module are: 

Van Dal en, Deobold B. Understanding Educational Respar rh. 
New York: McGraw-Hill, 1966. ~ 

Woodbunv, Marda. "A Guide to Educational Resources," in The 
m""^""' ?"!^°""^tion Consultant: Skill, in DisseninatiT^ 
Etetm.<aMnTonnMim. Berk eley, Calif.: Far West Laborato ry 
for Educational Research and Development, 1971 

Burke, Arvid J. & Mary A. Documentati on in Educati on. 5th ed 
New York: Teachers College Press, 1967 ^Hi^ ea. 

Sawin, E.I. "Subjective Influences in information Searches " 
A paper prepared specifically for this .nodule. August 1971 ! 

Marron, Harvey & Sullivan, Patricia, "Information Dissemination 
in Education: A Status Report MUgejLnlJMyersity Research 
Libraries. July 1971, pp. 286-294 — ^ Kes>.arcn 
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Grimes, G. & Doyle, =.1. Information ResourrP*; ; A Searcher's 
Manual. Detroit: Michlgan-Ohio Regional Educational Laboratory, 

David. ILlJ^^iyU.^"^"'"^'""'^ Research. An Tn...H....,-.. ^ew York: 
END-OF MODULE ASSESSMENT TECHNIQUE : 

Devices and procedures for checking on student progress are built Into the 
learning episooes for the module. Provision Is made for assosng both the 
of stuZ?s "'"'^"^ conducting a search, and actua? search smis 

COMPETENCES THE MOD ULE IS l^ lS iGNED TO ENHANCE: LEVEL OF COMPETENCF 

?nnH^"i-^ 'fS""3tion Search of the Problem Area Low Prof. 

A- .4 Conductino at. Information Search of the Problem Area Low Prof 

B- .1 Designing an Infomation Search (Development) iZ P?Sf 

B- .2 Conducting an Information Search (Development) Low Prof 

C- .1 Designing an Infomation Searcn (Evaluation) Low Prof 

C- .2 Conducting an Information Search (Evaluation) Low Prof 

n Vl ^" Information Search (Dissemination) Low Prof! 

D-1.2 Conducting an Information Search (Dissemination) Low Prof 



2- Information/Data Conection and Organization 



MODULE: 2.6 

The Retrieval of Informat i on 



Using Special Sources 



OVERVIEW: 



The purpose of this module is to acquaint the student with various soecial 

fH'°"r^" *° ^^^^^'^^^ instruction i^ Se usf o^ oL such 

system, the Educational Resources Infonnation Center. The student will be 
introduced to the techniques for locating and retrieving infomaU on documents 
on a specific topic and will be provided with experience in searching fS? 
documents, abstracting them, and organizing and preparing a TreselllStion on the 

OBJECTIVES: 

The overall objective of this module is to acquaint the student with several soecial 
information sources. Particular activities will be designed to help develop ^ 
competence in using one such system, the Educational Resources lS?o?ma?ion Center 

1. Is familiar with a variety of special infonnation resources. 

^' DATRIX*^' '""^ °^ information available through 

b. Explain how to locate and obtain documents tbr^- MATRIX 

thrSugh^ALERl! '""^ ^^^^ ^"^^""^tion available 

d. Explain how to obtain and use the ALERT system. 

2. Know the general structure and types of infonnation stored in the ERIC System. 

a. State the full name of ERIC and describe how it .s organized. 

b. State the function of the ERIC clearinghouses. a^^^^ea. 

c. Describe the types of infonnation documents stored in the ERIC System 

nZ'ln"^ fn' r-'^""' ''^'''"^^ "^^^ ERIC system by 

name and function. j j 

3. Is familiar with the procedure for procuring documents stored in the ERIC system. 

^' iJldudinrpricef'^"'"^ ordering hardcopy of ERIC docnjments, 

b. Explain the procedure for ordering microfiche copies, including prices. 

4. Know how the various ERIC reference tools are organized. 

a. Stats the content and. organizational structure of Research m Education. 

b. Locate entries in RIE by subject, author and institution. 

«Jl7'dLrHn'Jn^c'" "^J^ '.j"^! ^^ing identifying numbers, codes, author, 
title, descriptors, and other information. 
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d. State the content and organizational structure of Current Index 
to Journals -in Education, { c.],^^:.) ~~ 

e. Locate entried in CUE by subject and author. 

f. Interpret entries in CUE, including identifying numbers, 
codes, author, title, descriptors, and other information. 

5. Know the purpose and function of the Thesaurus of ERIC Descriptors . 

a. State the function and organization of the Thesaurus . 

b. Interpret the symbols and terms used in the Thesaurus . 

c. Use the rotated descriptor display, descriptor group display, 
and descriptor group scope notes to identify ERIC descriptor's 
appropriate to a given topic. 

6. Know how to complete a search of the ERIC system, using the various 
reference tools. 

a. Given a topic, develop appropriate and comprehensive list of 

descriptors using the Thesaurus . 
■ b. Locate potentially relevant entries in RIE and CUE. 

c. Ascertain relevancy of located entries by reviewing abstracts. 

d. Retrieve selected documents, view microfiches and prepare 
summary >'eport of selected relevant documents. 

7. Know the general function of the ERIC computerized retrieval systen.. 



a. Describe the function of DIALOG. 

b. State the general procedure for retrieving infcrmation 
through the DIALOG system. 

DESCRIPTION OF CONTENT: 

This module is divided into six learning episodes: 

.1. Introduction to Information Storage and Retrieval Systems 

2. Introduction to ERIC 

3. ERIC Reference Tools 

4. Use of Descriptors in Information Retrieval 

5. Conducting a Search' using ERIC Reference Tools 

6. Preparation of a Search Report and Introduction to Dialog 

Each learning episode is self-contained and consists of reviewing various 
documents, films, slides, film strips, and using the ERIC System to locate 
and retrieve documents. This module has no assigned text. Rather, there is 
a series of documents, pamphlets, instructional manuals, excerpts from publicatic 
and audiovisual aids in a supplementary booklet which accompanies this module 
The major activities of this module-the location and retrieval of information- 
will be accomplished by the use of the Thesaurus of ERIC Descrip tors and volumes 
of Research In Education and Cu rrent Index to Journals in Education The 
various instructional materials are organlTed by exhibit number in the module. 
Other resources and reference material that needs to be available for student 
use are one film,, three film strips, one slide presentation, current copies of 
RIE and CUE and the Thesaurus. Also a microfiche collection and viewer will be 
needed. mii 



A. Articles; 

1. Burchinal, Lee G., "The Role of the federal Government in 
Information Systems in Education. "Journal of the American 
Society for Information Service, July-August, 1970, pp. 274-278. 

2. "Processing the Center's Collection," Reference Manual for 
Educational Information Service Centers, Chap. IV, Falls Church, 
Virginia: System Development Corp., (Technical Memorandum 
TM-WD-521/100/01 , prepared for the U.S. Office of Education), 
1958. 

3. "Information Systems and the Teacher of Teachers," Robert Harvey, 
"NTCE/ERIC, Champaign, Illinois. 

4. "Teaching ERIC," Robert C. Harvey, NCTE/ERIC, Champaign, 111. 

B. Pamphlets : 

1. Information Pamphlet and Examples on ALERT," Far West Laboratory 
for Education Research and Development, 1970. 

2. Information Pamphlet and Examples on DATRIX, Xerox Corporation. 

3. PaMphlet: "How to Use ERIC" Office of Education, U.S. Dept. of 
Health, Education and Welfare. U.S. Gov't Printing Office 
0E-12037-C. 

4. "What's So Special About ERIC at Stanford?" Clearinghouse Brochure 
No. 1. 

C. Newsletter 

T. ERTC~at Standord 

D. Booklet 

T. ^Guide to Educatipnal Resources" preliminary form. 

Far West Laboratory for Educational Research and Development, 1971. 

E. Document 

1. ERIC Document ED 036 499, "How to Conduct a Search through 
ERIC" ERIC Clearinghouse on Teacher Education, No. 1 Dupont 
Circle, N.W., Washington, D.C., 1970 (with microfiche) 

F. Film & Fi l m Strip s 

1. ' Spires'/Ballots Report, #7789. (15 minutes) U.C. Extension Media 

Center, Berkeley, Calif. 94720. 

2. "Introduction to ERIC", National Audiovisual Center, Washington, 
D.C. 20409 (Set of 3 filmstrips and accompanying record, $5.00). 

3. Slide presentation: Interactive Information Retrieval from ERIC 
at Stanford, 27 slides. 

G. Indexes: 

1. Research in Education. 

2. Current Index to Journals in Education 

3- Thesaurus of ERIC Descriptors , CCM Information Corporation, 
N.Y., 1970. 
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H. DIALOG Readings: 



1 . Terminal Users Reference Manual for DIALOG , Information 
;>ciences, Lockheed Palo Alto Research Laboratory 

2. Examples of DIALOG print-out title sheet. 

3. Kxamples of DIALOG print-outs of sunmaries. 

4. Summary of Report: An Interactive Information Retrieval System-Case 
studies on the Use of DIALOG to Search the ERIC Document File, 
Michele Timbie and Don Coombs, ERIC Clearinghouse, Stanford, Ca., 
Dece mber 1969. 

END-OF-MODULE ASSESSMENT TECHNIQUE: 

The student will select a topic, develop descriptors, search ERIC publications, 

identify potentially relevant documents, review abstracts, retrieve and review 

microfiche of relevant documents and select, organize, and prepare a written 

presentation, providing in the report titles and abstracts or ERIC documents 
appropriate to the topic. 

COMPETENCES THE MODULE IS DESIGNED TO ENHANCE : LEVEL OF COMPETENCE 

A-1.3 Designing an Information search of a Problem Area Low Prof. 

A-1.4 Conducting an Information Search of a Problem Area Low Prof. 

B-1.2 Conducting an Information Search (Development) Low Prof . 

C-1.1 Designing an Information Search (Evaluation) 1-ow Prof. 

•C-1.2 Conducting an Information Search (Evaluation) Low Prof. 



3.0 COMMUNICATION Siaili 



The Communication Skills series is presently undergoing extensive revision. 
However, a general outline of the Series Is described reflecting present plans 
to develop four modules. 



MODULE 3.1 

Listening and Speakincj 



OVERVIEW 



This module will focus on the- ways in which formal organizations com- 
municate Internally. The aim of the module will be to develop student 
skills in receiving, organizing, and transmitting information in an organi- 
zational setting. ^ 

OBJECTIVES 

1. Will be able to understand assignments, instructions, and directions and 
to obtain answers to specific questions. 

2. Identify the accuracy or the bias (slant) in the content of speeches. 

3. Be able to respond to telephone messages and give adequate directions 
over the phone. 

4. Use information to direct additional data gathering efforts for the 
purpose of Identifying likely consequences of alternative actions. 

DESCRIPTION OF CONTENT 

The need to understand, respond to, and follow spoken instructions, 
answer questions, or interact with groups and individuals is universal; 
therefore, while the materials of the module is specifically designed 
around the type of work encountered in a DD&E agency, the module components 
can apply to any field of work. Audiotapes, transcripts, and role playing 
will be employed to structure exercises. Sample tasks may include situations 
Involving receiving and giving oral instructions, participating in and 
reporting on staff conferences, and conducting informal briefings and 
demonstrations. 



3. Comnunlcation Skills 



OVERVIEW 

Adequate preparation of technical documents is an essential skill in 
any DD&E effort. Therefore, this module will cover skills required for 
preparation of written forms which provide direct guidance for or are 
themselves the direct products, e.g., scenarios, scripts, instructional 
oests, teachers manuals, and test instruments. The module will provide 
opportunity for students to assist in the simulated development and 
evaluation of the technical documents similar to those listed above. 

OBJECTIVES 

1. Will be able to identify the purpose and design of a sample script. 

2. Will be able to evaluate and criticize a sample script. 

3. Will be able to identify the major processes involved in developing a 
simple test instrument. 



MODULE 3.2 
Technical Writing': 
GuTdance an?' ^ 
Instructional Material s 



^' test ^^^^ *° prepare a set of test instructions for a simple objective 

5. Will be able to produce a short scenario or script. 
DESCRIPTION OF CONTENT 



Episodes in this module require the student to construct, evaluate and 
analyze documents such as scenarios, scripts, and instructional tests and 
teachers manuals. Each learning episode will deal with the design and 
construction of the above mentioned products. Models and special exercises 
will provide the student with practice in designing these products 



MODULE 3.3 
Technical Writing; 
Work Support 

3. Communication Skills Documentation 
OVERVIEW 



This module will emphasize the competencies needed to write supporting 
instructions and documentation of work progress. These include: memos, 
outlines, progress reports, situation papers, literature reviews, field 
test instructions, and evaluation reports. 

OBJECTIVES 



1. Will be abla to identify supporting documents and their common uses 
in DD&E agencies-. 

2. Will be able to prepare simple memos and outlines. 

3. Will be able to list major procedural rules in preparing progress and 
evaluation reports. 

4. Will be able to prepare a brief literary review and anotated 
bibliography. 

5. Will be able to identify steps necessary in designing field test 
instructions. 

6. Will be able tc prepare simple user instructions. 
DESCRIPTION OF CONTENT 



The episodtvi in this module will include a number of sample support 
documents to familiarize the student with the various types and uses of 
such documents. Student activities will require the student to prepare 
short formal reports (e.g., position paper, technical report, or task 
planning paper) using standard formats and report styles, footnotes, 
outlines, and bibliographies. 



3. Communication Skills 



MODULE 3.4 
Technical Writing: 
Formal Publications 



OVERVIEW 

This module is designed to familiarize' the student with formal 
communication skills and procedures such as the preparation of formal 
papers and publications, e.g., proposals, final reports, and journal 
articles. 

OBJECTIVES 

1. Will be able to recognize the use of several formal documents. 

2. Will be able to identify the various methods used in preparing 
formal papers and reports. 

3. Will be able to explain the basic format of journal articles. 

4. Will successfully assemble and complete sample formal documents. 
DESCRIPTION OF CONTENT 

The format requirements and literary quality of journal articles 
will be stressed with the trainee being orovided examples to critique, 
edi^, and evaluate. Exercises requiring the trainee to conduct a 
review of the literature and develop a summarization of a potential 
journal article. Opportunities to critique formal reports will be 
provided. Criterion models of reports will allow students to evaluate 
their own critiques. Other tasks will involve the use of pertinent 
references dealing with style and composition. 



4. Developmental Engineering 



MODULE: 4 J 

Establishing Development 
Objectives 



OVERVIflW: 

This module serves two functions: to provide an orientation to the field of 
Developmental Engineering and to assist the student in acquiring skills in 
establishing development objectives. Upon completion of the module, the 
student should be ready to begin development of an educational product, or 
component of a product. 

OBJECTIVES: 

By the end of the module, the trainee will be able to develop: 

1) an analysis of an educational problem 

2) a specification of a solution to that problem 

3) a description of a product included in that solution 

4) a detailed design for a component of thnt product 

5) a plan for developn^}nt of that component 

6) a statement of development objectives for that product 
Linkage with other Modules: 

These modules (1.1, 1.2, 1.3) will give the trainee background information which will 
be helpful in working through this Module. 

1.1 The Specification of Expected Outcomes 

1.2 Consideration of Alternatives 

1.3 Planning for Development 

These Modules (6.3, 6.4 ) could be taken concurrently with this Module 

6.3 Problem Formulation and Specification 

6.4 Analysis of the Formulated Problem 

The following Modules ( 1.6, 1.7, 4.2) logically follow this Module. 

1.6 Introduction to Component Design 

1.7 Planning for Component Design 
4.2 Engineering a Component 



Description of Content ; 

This module is made up of five learning episodes divided into -19 activities for 
the student. The activities are based on a series of assigned readings which 

!I!;4*-!4^*°"f development of a variety of educational products, and 

the activities and problems associated with their development. 

iHJfllrJu \\ student reviews the factors to be considered in determining a 
product through careful analysis of a problem statement and of the characteristics 
of a potential solution to the problem. The second episode shows how to 
translate a general idea for a solution into a set of specifications from which 
a product or component of a product can be developed. Episode 111 focuses on 
planning for product development - determining what tasks and resources will 
?hir''"JMf J° ?!;°-"'^ ^"^ developing a schedule for accomplishina 

i^n'.'.^f 5 ? aesigned to acquaint the student with some of the issues' 

in ^nHMn^t ^^^^ w^^^ ^^eir work. it should assist the students 

rilfi^n^ISJ ^""J^^SH^ i"?"^ problems which may arise in their own product 
development work. In the final Episode, the student is presented with a 
?H !!-;^°Ju term development objectives, their function and how to 

Identify the kinds of background information necessary to establish such 
objectives. As a summative activity, he is given a specific product or 
component and asked to state the objectives for its development. 

The needed reading materials appear exactly where they are called for However 
a reference to the complete report (or other material) from which the reading 
has been extracted appears in the bibliography. There are no outside reading 
assignments. u«i.o.uc .caumy 

The Readings Used in the Module are: 

Appal achia Educational Laboratory. "Ntodel for Educational Development, 
an abstract." Charleston, West Virginia. (Appendix 1) 

Bialek, Hilton. "Design and Development of a Simulated Game as an 
Instructional Device for Training Educational Disseminators - Diffusors " 
hxcerpt from a draft of a proposal prepared by the Human Resources 
Research Organization, Division No. 3, Monterey, California (Appendix K) 

Borg, Walter, R. "The R&D Process as Used in Designing 
Mini courses." Far West Laboratory for Educational Research 
and Development, 1968. (Appendix 1) 

Dunn, James A. "The PLAN Guidance System - 1970." Operations 
Council Position Paper, Dec. 11, 1969. (Appendix E) 

Hood, Paul D., et.al. "Developmental Engineering Course," excerpt 
1^.°^ Design of a Functional Com petence Training Program for Dev eloompnt 
Dissemination, and Evaluat ion Personnel at ^Professional and Par aorofessional 
^H!r]f^nn ^f""'^^'P" -' Vol "Hie two , Secion G. Berkeley: Far West Laboratory for 
Education Research and Development, 1970. (Appendix G) 
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oenks, Lynn. "Training Unit" Objectives and Evaluation, a 
Concept Paper." in draft. Far West Laboratory for Educational 
Research and Development, 1971. (Appendix C) 

Kratochvil, Dan. Sesame St reet. Technical Report #10. Palo 
Alto: American Institutes for T^search , 1971.( Appendix F) 

Smith, Robert G., Fr. "Models of the Engineering Process," excerpt 
from The Engineering of Educational and Training Systems . Lexington, 
Mass.: Heath Lexington Books, 1971. (Appendix 1 ) 

Southwest Educational Development Laboratory. "Introduction" and 
"The Concept of Product Development," excerpts from A Development 
Process Adop ted by the $outhwest Educational Development Laboratory 
July 14, 1970. Southwe<,t Educational Development Corporation: T970. 
(Appendix J) 

Thompson, Lorna and Kratochvil, IJan. The Creative Learn in g Group Drug 
Education Program. Technical Report #5. Palo Alto: ~" 
American Institutes for Research, 1971. (Appendix D) 

Supplementary Readings are.- 

Dillman, Frederick E. Jr. Instructional Objectives: Specificity and 
Behavior. A m'li-program from the Institute of Communication Technology, 
nemo Kark: Dillman Assoc., 1971. 

Handy, H.W. and Hussain, K.M. Network Analysis for Educational Management. 
Engle Cliffs, N.J.: Prentice-Hall, 1969. 

Mager, Robert F. Preparing Instructional Ob.iectives . 
Palo Alto: Fearon Publications, 1962. 

Enrichment and Supplementary Activities : 

There are no specific supplementary or enrichment activities given in the module. 
Student Assessment Techniques _ 

There are six progress checkpoints in this Module. In the 1st checkpoint the student 
IS asked to prepare a statement which identifies the components which should 
be included in an analysis of an educational problem. In the 2nd checkpoint the 
student prepares a statement evaluating the strengths and weaknesses of the 
alternatives (developed in checkpoint 1). Based on this evaluation the student 
selects the best solution and defends the selection. In the 3rd checkpoint the 
student prepares a written description of the selected product. The student then 
designs one component of the product. In the 4th checkpoint the student prepares a 
schedule for the development of the component. In the 5th checkpoint the student 
reviews the schedule and looks for potential decision or problem points and then 
makes any needed revisions in the developmental plan. In the 6th checkpoint the 
s vudent specifies the development objectives for. the product to be developed 
in the subsequent modules of this series. 

There wi'i. be no end-of-module test for this module. 
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Competence Elements the Module is Designed to Enhance: LEVEL OF COMPETEN CE 

A-1.1 Identifying the Problem Area Familiarization 

A-4.1 Specifying the Desired Outcomes Familiarization 

A-7.1 Preparing the Description of Problem Analysis Familiarization 

B-3.1 Establishing Development Objectives Familiarization 

B-3.2 Preparing a Development Plan Familiarization 

B-3.3 Determining Developmental Parameters Familiarization 

B-3.5 Sclieduling for Production Familiarization 



4* Developmental Engineering 



Module: 4.2 
Engineering a Component 



OVERVIEW : 

This Module instructs the student in instructional systems theory 
and the design elements of a component or product of an instructional 
system. Further it identifies the sequential process for assembling 
these elements* 

OBJECTIVES : 

The overall objective of this module is to enable the student to 
create and assemble a prototype, instructional component. 

There are three groups of sub-objectives which will enable the 
student to attain the main objective. These are: 

1. to understand the major concepts and steps in designing 
an instructional system, 

2. to understand the concepts and design elements necessary 
to create an instructional product and to engineer a 
component, 

3. to prepare an actual prototype component from a set of 
design specifications. 

LINKAGE TO OTHER MODULES : 

These modules (1.6, 1.7, 4.1) will give the trainee background 
information which will be helpful in working through this module. 

1.6 Introduction to Component Design 

1.7 Planning for Component Design 

4.1 Establishing Development Objectives 

4.3 logically follows this module. 

4.3 Building A Whole Out of Its Parts 

' DESCRIPTION OF CONTENT : 

The module is divided into two episodes. The first episode is 
optional for those students without an extensive background in the 
theory of instructional systems. 
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The first episode introduces the key concepts and key design elements 
in creating an instructional system. In the s3Cond section of this 
episode, a case study is presented which illustrates how a team of developers 
went about creating an instructional system. 

The second episode focuses upon the concents and design elements 
leading to the creation and engineering of an instructional product and 
comoonent. The episode concludes with a return to the case study and 
investigates how the team of developers prepared the products for the 
system they were designing. 

ENRICHMENT AND SUPPLEMENTARY ACTIVITIES: 



None. 



STUDENT ASSESSMENT TECHNIQUES : 



There are two Progress Checkpoints in this module. In the first 
Checkpoint the student is asked to describe, in sequence, the steps used 
to design an instructional system, and to recognize examples of these 
steps from reading material. In the second Checkpoint the student is 
asked to a) list the inputs needed to design a product, b) describe the 
decisions necessary to design a product, c) describe the difference 
between a component and an instructional product, d) list the elements 
of an instructional product, 3) describe schedules and checklists 
necessary in the design of an instructional product, f) recognize the 
necessary ingredients of an instructional product from a reading. 

The End of Module Exercise asks the student to create an instructional 
product component using given sample component requirements. 

COMPETENCE ELEMENTS THE MODULE IS DESIGNED TO ENHANCE: LEVEL OF COMPETENCE 



B-4.1 Developing a Component 



Specify the purpose of a given component 
of the instructional system. 
Specify the performance outcomes or 
objectives. 

Specify constraints and resources. 
Select instructional content. 
Organize and sequence instructional 
content- 
Select methods and media for 
presentation of instructional content. 
Prepare tests and measures to assess 
student performance. 
Evaluate effectiveness of component or 
product in meeting performance outcomes. 
Modify the component or product to 
improve its effectiveness in meeting 
performance outcomes. 



Low Proficiency 

Familiarization 

Familiarization 

Familiarization 
Familiarization 
Familiarization 

G mentation 

Familiarization 

Orientation 

Orientation 
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5. Evaluation MODULE: 5.1 

The Role of Evaluation in 
DD&E 



OVERVIEW : 

This module will introduce the student to a few basic concepts and the 
general functions of evaluation in the different contexts of D,D&E. 
This orientation material will be received and then several case studies 
will be examined to provide meaningful introduction to the many facets of 
evaluation.. 



OBJECTIVES: 

1. Present and. describe examples of several evaluation efforts. 

2. Explain the general purpose of performing an evaluation. " Present 
examples of specific evaluation objectives. 

3. Outline the basic steps in the evaluation process. 

4. List many of the day-to-day tasks performed by professionals involved 
in an evaluation. 



5. Define and give examples of the following terms. 

val idity 
reliability 

statistical significance 
formative evaluation 
summative evaluation 



DESCRIPTION OF CONTENT: 

This module includes five case studies. One deals with v^yious evaluations of 
the television program Sesame Street. The case studies are the real heart 
of this module. Explanatory text has been kept to a minimum. The module 
IL Hn«nnc''I!*° t^^V^ sections. Section 1 describes the purpose of evaluation 
and defines two kinds of evaluation. Secion- 2 is the case study on the 
central city school program. Section 3 outlines the basic steps in the 
evaluation process and defines some of the important terms. Sections 4 
through 7 contain case studies dealing with the evaluation of Sesame Street. 
Section 8 contains several final activities. 

The references are: 



Ball, S. and Bogatz, G.A. A summary of the ma.ior findings in "The 

first year of Sesame Street; An eval uatinn." PHnrptn^ 

Educational Testing Service, 1970. 
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Cook, D.L. Program Evaluation and review technique: applications in 
education, U.S. DHEW OE; OE-1 2024, Cooperative Research Monograph 
No. 17, Washington D.C.: U.S. Gov't Printing Office, 1966. 

Cronbach, L.J. Es sentials of psychological testing. 2nd ed ition. 
New York: Harper and Row, 1960. 

Cronbach, L.J. Course improvement through evaluation. Teachers 
College Record . 1963, 64 (8), 672-683. ' — 

Grobman H. Evaluation activities of curriculum projects. AERA 
monograph^series on curriculum evaluation. Chicago: Rand McNally 

Hawkridge, D.G., Campeau, P-.L. & Trickett, P.K. Preparing evaluat ion 
repcrts; a guide for authors, Pittsburgh: American Institutes for 
Research, 1970. AIR Monograph No. 6. 

Light, R.J. & Smith, P.V. Choosing a future: strategies- for designing 
and evaluating new programs. Harvard Educational Revie".-.' . 1970, 40(1), 
1"28* 

Phi Delta Kappa National Study Committee on Evaluation. Educat ional 

evaluation and decision making. Tta^ra. Illinois: F.E. Peacock 

Publishers, Inc., 1971. 

Scriyen, M. The methodology of evaluation. AERA monograph series on 
curriculum evaluation 1: perspectives of curriculum evaluation. 
Chicago: Rand McNally & Co., 1958. 

Suchman, E.A. Evaluative research . New York: Russell Sage Foundation, 

END-OF MODULE ASSESSMENT TECHNIQUE : 

In Section 8 of the Module the student is asked to: 

(a) compile a list of tasks professionals actually 

do from day to day when they work on an evaluation. 

(b) outline an evaluation plan of a product 

COMPETENCES TH E MODULE IS DESIGNED TO ENHANCE: LEVEL OF COMPETENCE 

C-1.3 Inventorying Evaluation Alternatives Familiarization 

ri 1 Sjalyzing the feasibility of Evaluation Strategies Familiarization 

ri*? V.lrVJ'J Evaluation Familiarization 

C-3.2 Specifying Types of Evaluation Information/Data Familiarization 

C-6.1 Assessing the Evaluation-Plan Familiarization 



5. Evaluation 



MODULE: 5.2 
Jest and Measures 



OVERVIEW; 

s udent with the critical properties of tests an iveexpe? ^ c „* ow to 
choose appropriate tests on the basis of these properties- (br intrnHurp th. 
se'S*an'S f^hZl llZZTulTlTA^' l^^^li^e^'^^ 
OBJECTIVES; 

At the conclusion of this module the student will be able to; 

1. describe the distinction between measurement and evaluation- 

2. specify and define the four levels of measurement; * 

3. define the meaning of the term "standardized tesn" 

4. define the meaning of "norm" group; 

5. distinguish among the various types of test validity; 

6. interpret the meaning of various typos of test reliability 

/. interpret the meaning of various methods of expressing test scores- 

a'stap'dard zL* '''/l''''' factors that shouldV considered ^ ' lecting 
e'du^'a'tn'al'proiuct';'" ^" "^'"^^"^ effectiveness of a develoje'd 

t'^^I^^^J^Z^^:^^'^ -^^-^^ their appropriateness 
DESCRIPTION OF CONTENT : 

This module is divided into six learning episodes. 

^' discussIs^Jpvfffn!*' introduction to tests and measures and 

2nd critir!? ^l^olc""'""'"'"*'' conditions of test administration, 
ijnd critical factors concerning norm groups. 

2. This episode introduces you. to the concept of test validity the 
various types .of validity, and how they are determined 

3. In this episode the various types of test reliability will be orespntpH 
d?sc5st5''''^""^'^"^ '''''' measurement and Jeli^M-li'ty 

^^^^t^r^^ ^-"terpreting them is 

^' Jis'cu'sslon'nJ^^h! f f ^"dardized tests are presented along with a 
discussion of the factors influencing the selection of tests. 
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The Readings in the Module are: 

Adams, G.S., Measurement and Evaluation in Education . Psychology, and 
Guidance, New York: Holt, Rinehard and Winston, 1964. 

Ahmann, J.S., and Glock, M.D., Evaluating Pupil Grow th.' 4th ed., 
Boston: Allyn and Bacon » 1971 . 

Anastasi, A., Psychological Testing. New York: The Macmillan Co., 3rd ed., 

Boyd, J.L. and Shimberg, B., The Directory of Achievemen t Tp<;t<; for 
Occupational Education . Princeton, New Jersey: Ed'icational Testing" 
Service, 1971. ' 

L^^''MontI'RIsearch1ark.^ ^^'""""^ Measurement Terms. Monterey, California, 

Johnson, O.J., and Bommarito, J.W., Tests and Measurement <; in Child 
Development: A Handbook, San Francisco, California: Jossey-Bass, Inc. 

Linden, J D., and Linden, K.W., "Judges and Juries," Tests on Trial 
Guidance Monograph Series 111: Testing. 

TT-rn '"I^st Publishers and Scoring Services," Tests on Trial 

Guidance Monograph Series 111: Testing — - 

Perry D., Interpreting Standardized Te<;t SrnrP<;. Minneapolis, Minnesora; 
University of Minnesota, 1971 ( technical Report #8000) 

The Psychological Corporation, " Methods of Expres sing Test Scores " Test 
Service Bulletin #48, New York, January, 1955 

The Sixth Mental Measurements Ye.->rhnnk, Ed. O.K. Buros, Highland Park," 
New Jersey: Gryphon Press , 1971 . 



The Readings in the Module are: 

Adams, G.S., Measurement and Evaluation in Education . Psychology, and 
Guidance,. New York: Holt, Rinehard and Winston, 1964. 

Ahmann, J.S., and Clock, M.D., Evaluating Pupil Growth . 4th ed., 
Boston: Allyn and Bacon, 1971. 

Anastasi, A., Psychological Testing . New York: The Macmillan Co., 3rd ed., 
1968. 

Boyd, J.L. and Shimberg, B., The Directory of Achievement Tests for 
Occupational Education . Princeton. New Jersey: Educational Testing 
Service. 1971. 

California Test Bureau. A Glossary of Measurement -Terms . Monterey. California. 
Del Monte Research Park. 

Johnson. O.J.. and Bommarito. J.W.. Tests and Measurements in Child 
Development: A Handbook. San Francisco. California: Jossey-Bass. Inc. 

Linden. J.D.. and Linden. K.W.. "Judges and Juries," Tests on Trial 
Guidance Monograph Series 111: Testing. 

."Test Publishers and Scoring Services." T ests on Trial 

Guidance sMonograph Series 111: Testing 

Perry D.. Interpreting Standardized Test Scores, Minneapolis. Minnesora; 
University of Minnesota. 1971 (Technical Report #8000) 

The Psychological Corporation. " Methods of Expressing Test Scores ." Test 
Service Bulletin #48. New York. January. 1955. 

The Sixth Mental Measurements Yearbook . Ed. O.K. Buros. Highland Park. 
New Jersey: Gryphon Press. 1971. 
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The Readings Used Which are Outside the Module: 

American Psychological Association, Standards for Educational and Ps ychological 
Tests and Manuals, Washington. D.C., 1966. ^ 

Borg, W.R., and Gall, M.D., Educational Research , an Introduction. 2nd ed , 
New York: David" McKay Co. , Inc. , 1971 . 

Levine, S., and Elzey, F., A Programmed Introduction to Educational and 
Psychologic al Measurement, Belmont. California: Brooks Cole Publishing 
Co. 1970. ^ 

The Optional Readings are: ' 

Engelhart, M.D., Methods of Educational Research. Chicago: Rand McNallv 
and Co., 1972. 

Sjogren, D.D., " Measurement Techniques in Education ." Review of Educational 
Research, Vol. #40, 11, April, 1970. 

END OF MODULE ASSESSMENT TECHNIQUE: 

The student is asked to select a standarized test that would be suitable for use 
as a pretest and posttest in determining the effectiveness of a specific 
product's use with the intended audience. 

COMPETENCES THE MODULE IS DESIGNED TO ENHANCE: 

C-4-.1 Preparing Evaluation Instruments 
C-5.1 Administering Evaluation Instruments 
C-6.2 Assessing the Evaluation Instruments 



5. Evaluation 



MODULE: 5.3 
Development of 
Instruments 



DESCRIPTION OF CONTENT 

This module is being redesigned and the specification of content is 
not yet complete. However, the module will provide trainees with infor- 
matton on the basic steps for developing evaluation instruments and will 
include exercises that require the construction of sample test items or 
small components of evaluation devices. 

FORMAT 

The form of the module will be print and will follow the structure 
of Module 4.2 (see Attachment Two). Special exercises will be designed 
that require the student to practice the design of test items and develop- 
ment of small test instruments including item analysis and testing for 
reliability and validity. A simulation exercise will be included in the 
standard test battery that will allow the trainee to demonstrate his or 
her proficiency in developing small test instruments. 



5. Evaluation 
DESCRIPTION OF CONTENT 



MODULE 5.4 
Field Tests 



Students will read case studies and supporting technical information 
about the conduct of field tests. Students will practice administering 
a variety of Instruments to each other under simulated field test con- 
ditions. Each student will then be provided with a file containing field 
test plans, descriptions, and analyzed data representative of the products 
of a field test. He will be directed to follow guidelines to evaluate 
the data and produce a short report of his findings. Students will study 
descriptions of the types of content analysis often used in development 
projects and perform at least one content analysis of actual data. The 
results will, be checked and revised until they are satisfactory. Case 
materials illustrating commonly encountered problems in the administration, 
analysis, and evaluation of instruments used in questionnaires, surveys, 
observation, field tests, etc., will be presented and discussed. 

FORMAT 

The format will be print and will contain four Episodes. Each episode 
will provide activities for the student to complete. The feedback section 
will give the answers to these activities. Also, there will be two or 
three progress checkpoints in which the student will do simulations and 
evaluate data froii; sample field test situations. 



5. Evaluation 



MODULE 5.5 
Evaluation Problems 



DESCRIPTION OF CONTENT 

This module will provide practice in coping with a variety of informal 
and formal evaluation problems-, using examples drawn from all phases of 
development from analysis through dissemination. Most of the problems 
presented at this level will focus on the judgmental and value problem of 
obtaining and bringing data and information to bear on the type of important 
decisions encountered throughout all development phases. The student will 
be introduced to basic approaches and methods for systematically examining 
one problem, the nature of the evidence needed, the adequacy of the evidence 




organizing and analyzing this information 
le systems. 



FORMAT 



The format will be print ajid will contain four episodes. The student 
will be required to do activities and progress checkpoints. Also, there 
will be an end of module test to complete. 



fJODULE 6.1 
Problem 

6. Analysis and Definition FormuVat ion 
DESCRIPTION OF CONTENT 

In this module, the student will begin with an overview delineatinq 
the function of analysis and problem definition in DD&E Agencies and the 
kinds of ski! s required. An episode will then raview system analysis, 
problem Identification, and problem definition including practice in 
establishing needs and identifying problems. Another episode will provide 

^d^nH?«^3!l'!^^°^fc'1K^"^ to specific problems. An exercise will involve 
Identifying and describing problem areas and examining approaches for 
analyzing problems in terns of their validity, critlcality, and feasibility 
of solution. A final exercise will be the preparation of a problem state- 
ment. 

FORMAT 

The format of this module will be print and will contain four episodes; 
the last Involving a student exercise that guides the student to the develop- 
ment of a format, structured problem statement. Each episode will contain 
checkpoints wnereby the student can evaluate his/her progress as well as the 
standard battery of test instruments administered upon completion of the " 
module. No special audiovisual equipment will be necessary. 



6. Analysis and Definition 
DESCRIPTION OF CONTENT 



MODULE 6.2 
Problem Analysis 



This module will provide exercises consisting of problem statements 
for analysis by the student. From this Information, the student will 
revise his own problem statement, if necessary, developed in the first 
mod-'jle. A second episode will Involve the student in applying information 
related to the Important variables 1n the problem statement. Exercises 
will include such problem activities as definition' of populations, relating 
alternative solutions to criteria, resources, and market variables. The 
final student exercise will direct the student through a problem analysis 
including a complete statement of the problem and outlining constraints, 
alternativf; solutions, requirements, and recommendations. 

FORMAT 

The form of the module will be primarily print, the structure involving 
three episodes, each requiring the student to perform. some prescribed 
activity or exercise. The module will include self-evaluation exercises or 
questions with feedback, the standard assessment battery, and a final exercise 
that allows the student to produce a well organized, valid analysis of a 
problem statement. 
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Dissemination and Marketing 



MODULE: 7.1 

Design and i;va1iiaHnn^nf_ 
D issemination and Marketinq 



Models 



OVERVIEW; 
OBJECTIVES: 

The student should be able to: 



1. 



Its 
ve 



Recognize three aspects of dissemination: 1 
scope. Its characteristics; and representati 
generalizations about it. 

Recognize strengths and weaknesses of today's 
educational communication network and be able to 
mnifi! ^"^^Pe'^spective to the conceptualization of 
models of educational dissemination. 

Relate the function of dissemination roles assumed 

utifSnn^J^^H^'^PJ^* °r ''"^'^ upon the 

utilization of educational practices, products, and 



Linkage to Other Modul<^«;: 



ERIC 



1.5 Planning for Dissemination precedes this module. 
The other three'modules in the Dissemination and Marketi 



module. 
Description of Content: 



ing series .-ollow this 



ii;d;f:;^"j?„^^™iKS?,°:,-;jn^ iiir'"^^ ^ ^^^^-t activities 

in the module. • ictivines are based on readings prasented 

mffiVe 6ml^^},?,ltVf iS iLrl'^'i^J dissemination process 
about dissemination. Th^ tudent is asked ?o in' v .h'"''"^"?-''^ generalizations 
educational dissemination goals by idlntifvJnn fSl^^ generalizations to specific 
and by suggesting steps ne?es rJVa h I^^" sSocmfJoal 



In Episode II the student reads a description of the components of today's educational 
corraiunication network and an evaluation of the network in terms of communication 
components in fields routinely influenced and modified by scientifically derived 
infomation. The student then reads a description of a hypothetical dissemination 
process that is successful and descriptions of two actual attempts to disseminate 
educational innovations which were unsuccessful. The student is asked to critique 
today's educational communication network in general and one specific component 
within the network. The student then reads a discussion of criteria for developing 
dissemination models and is asked to identify from a prior reading the dissemination 
problem and the variables and processes involved in disseminating the innovation 
described. He then examines several dissemination models of the same process and 
constructs a model from a new descriptive reading. 

In Episode III the student reads two discussions of roles people play in the 
dissemination process and a description of a hypothetical dissemination program. 
The student is asked to identify the roles and functions of various people 
involved in the hypothetical program. The student then reads a discussion of 
various dissemination strategies and an excerpt from an actual marketing study 
for an educational Innovation. As a final checkpoint activity, the student is 
asked to discuss the marketing study in terms of rationale for identifying 
data needed, consideration of alternative distribution modes and the influence 
of data gathered on the choice of distribution mode. 

Excerpts from the following outside sources are incorporated in the module: 

Hutchins, C.L. . Mclntyre, D., Mitchell, !C.& Harding, J. Marketing 
study: Management training units, Berkeley, California: Far West 
Laboratory for Educational Research and Development, 1972. 

Louis, K.S. & Metzger L. Measuring the goals of actio: programs. 
New York: Bureau of Applied. Research, Columbia University, 1971. 

Rogers, E.& Shoemaker, F. Communication of innovations . New York: 
The Free Press Division, Collier-Macmillan, 1971. 

Travers, R.M.W. A study of the relationship of psychological 
research to educational practice. In R. Glaser (Ed.), Training 
research and education . New York: John Wiley and Sons, 1962. 

Suggested Supplementary Resources 

BOOKS. 

Bennis, W.G. , Benne, K.D. & Chinn, R. The planning of change. New York: 
Holt, Rinehart and Winston, 1961. (in FWL Library) 

Crawford, J.J., Kratochril, D.W. & Wright, C.E. Evaluation of the impact of 
educational research and development products. Palo Alto: American Institutes 
for Research, 1972. (in FWL Library) 
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Havelock, R- Planning for innovation. Ann Arbor: Institute for Social 
Research, University of Michigan, 1971. (in Learning Lab.) 

Miles, M. (Ed.) Innovation in education . New York: Teachers College, 
Columbia University Press, 1964. (in FWL Library) 

Miller, R. Perspectives on educational chang e. New York: Appleton, 
Century Crofts, 1967. (in FWL LibraryX 

Riccio, A. & Cyphert, T. Teaching in America . Columbui. Ohio: Charles 
Merrill and Company, 1962^ (out of print, in S.F. State Library) 

Rogers, E. & Shoemaker, F. Communication of innovations. New York: 

Free Press Division, Collier-Macmillan, 1971. Tin Learning Lab.) 



FILM: 



Hake a Mighty Reach , a film on educational innovations produced by 
the Institute for the Development of Educational Activities (IDEA) 
of the Charles F. Kettering Foundation. Dayton, Ohio, 1967. 16 mm 

This film may be rented from IDEA,Dayton, Ohio, for $1E; for three days. 

Otherwise it is available to S.F. State instructors only, to be viewed 
on campus. The film may be reserved by calling the Audio-visual Center, 
Tel. 469-1492, x 7038. 

Tel. 469-1494 to reserve projector and screen. If you need to reserve 

a viewing room at the audio-visual center, make reservations far in advance. 

Student Assessment Technigues: 

Progress Checkpoints 

There are five Progress Checkpoints in this module which lead the student from 
analysis of dissemination problems in general to evaluation and building of 
actual dissemination models. 

(Ep. I) 1. Progress Checkpoint 1 requires the st«'dent to identify 

innovations and target audiences in a variety of goal 
statements. It further requires the student to apply 
dissemination generalizations to one goal statement by 
suggesting steps necessary to achieve the goal. 

(Ep- 11) 2. Progress Checkpoint 2 requires the student to write a 

critique of today's educational communication network 
in general and one specific component within the 
network. For his network critique, the student is 
to use as evaluation criteria seven practices found 
in fields which are influenced and modified by 
scientifically derived information. 



He is to discuss the specific component in terms of: 
institution base, dissemination formats used, roles used 
to expedite diffusion and the nature of feedback sought. 

The student is also asked to identify t he strongest and 
weakest components of the educational communication's net 
work and c ontrast them in terms of: institutional base, 
dissemination formats used, roles used to expedite diffusion 
and the nature of feedback sought. 

(Ep- n) Progress Checkpoint 3 

The student is required to read a short description of the 
dissemination of an educational product and to construct 
a dissemination model based on the reading. The student is 
also asked to write down his criteria for structuring the 
model and the variables and processes included. 

Finally, the student is required to write a critique of his 
own model, noting 5 strengths and weaknesses. 

(Ep. Ill) Progress Checkpoint 4 

The student is required to write a critique of a wine service 
dissemination project which is described in Episode 3. The 
student is asked to discuss four steps in the dissemination 
process: 1. methods of dissemination used; 2. hovi the 
dissemination methods were focused upon targed audiences; 

3. innovator interaction with the targeted audiences; 

4. kinds of feedback sought by the innovator. 

The student is also asked to identify important aspects of the 
dissemination process which were neglected or poorly handled " 
and offer a course of action which might improve the overall 
dissemination effort. 

(Ep- ni) Progress Checkpoint- 5 

The student is required to write an analysis of a marketing 
study described in Episode 3. The analysis must include: 
1. a rationale for identifying data needed; 2. an overview 
of alternative distribution modes considered; 3. a discussion 
of how data influenced the choice of distribution mode. 

End of Module Test 

1. Application of Model Building and Evaluation Skills 

The end of module test requires the student to apply the dissemination concepts and 
model building and evaluation skills he has learned to three different educational 
dissemination problems. The student must be able to do the following: 



2. 



Identify target audiences for given innovations* 
Outline strategies for dissemination of given innovations 



3* 



List criteria to be used to determine best methods of 
dissemination from available alternatives* 



4* 



List variables that must be considered in order to 
effectively disseminate an educational product. 



2- Integration of Knowledge about dissemination as a Process and Problems of 
Educational Dissemination 

The student is also asked to discuss why unproven innovations are routinely 
diffused to educators and routinely adopted by them. (This question draws 
on information gained in Episode II). ^ 



Finally, the student is asked to outline a plan to resolve the problem of "diffusion 
and adoption of unproven educational innovations. 




Competence Elements the Module is Designed to Enhance: 



Level of Competence 



D-2.1 Assessing the Customer/Market 

D-2.2 Analyzing Marketing Feasibility 

D-2.3 Analyzing Marketing Compatibility 

D-2.4 Analyzing Marketing Cost-Ef^activeness 

D-3.1 Designing the Marketing Strategy 

D-3.2 Designing a Marketing Component 

D-3.3 Scheduling Dissemination Activities 



Fami 
Fami 
Fami 
Fami 
Fami 
Fami 
Fami 



liarization 
liarization 
liarization 
liarization 
liarization 
liarization 
liarization 



7. Dissemination and Marketing 



MODULE 7.2 
Working with the 
Consumer 



DESCRIPTION OF CONTENT 

(This module is now being redesigned; therefore, only a siuimarization 
of the content is provided). 

The module will address the following^ competence subsets: 

D2 Analyzing Alternative Dissemmination Strategies 

D4 Developing Marketing Devices 

D5 Implementing the Dissemination Strategy 

D7 Communicating Marketing Results 

An episode will be designed to discuss how to relate to and utilize 
communication resources offered by USOE-supported agencies, private 
foundations, professional associations and the world of business. 

A second episode will investigate what is known about current dif- 
fusion and utilization practices within the field of education, including 
studies by Wolf and Fiorina, Brickell, and Carlson. 

A third episode will discuss how to profit from what is known about 
current marketing practices within the field of education. 

FORMAT 

Exercises will be included to allow students to gain the competencies 
to the proficiency levels specified. The standard assessment battery will 
also be developed for this module. The module will be primarily textual 
with simulated exercises and self-evaluating checkpoints. 



MODULE 7.3 

, . Installing Educational ' 

I' PTSsemtnatiori and marketing Products 

DESCRIPTION OF CONTENT 

„f . (This module is now- being re-des1gned; therefore, only a summarization 
of the content is provided.,) 

The module will address the following competence subsets: 

D2 Analyzing Alternative Dissemination Strategies 
D5 Implementing the Dissemination Strategy 
D7 Communicating Marketing Results 

h is proposed that an episode focus on selecting the innovation to 

be diffused Including need analysis, cost-benefit considerations and 
resource capabilities for change. 

will h^'VL^ ^^^^ ^^^^ selecting the target audiences. This 

will include marketing surveys, sociometric analyses.;- and demographic data 

Episode four will address measurement and evaluation techniques. This 
will include cost-benefit considerations, availability of data, sources of 
data, and analytic capabilities. ^'jun-eb ui 

v.oun-«!l/^"^^ episode will deal with selecting feedback mechanisms, again 
revolving cost-benefit considerations as well as feedback usage. 

FORMAT 

This niodule will be structured in five episodes, each with practice 
exercises, text-Imbedded questions and self-evaluative checkpoints. The 
standard assessment battery will be developed including, 1f appropriate to 
the learning situation, a short simulated activity. The form of the module 



8. Management 
DESCRIPTION OF CONTENT 



MODULE 8.1 
Manaqernsnt of 
Personnel 



(The content of this module has not yet been specified; therefore, 
a summarization from the Design Report is provided.) 

The first episode in this module will focus on competencies involved 
in acquiring new employees. I.e., preparing job descriptions, interviewing, 
orientation, and assignment. 

Other eptsodes will focus on the management of personnel including 
morale, evaluation, employee efficiency, work planning, supervision, humar 
relations, and leadership. 

FORMAT 

The module will Involve limited simulation exercises where the 
student can react to structured situations and compare the response to 
models provided. Further, self-evaluation questions and checkpoints will 
be included, in addition to extensive readings on personnel management 
theory. The standard battery of assessment devices will be available. 




MODULE 8.2 

Q Mar,a«««n^ 4- Management of Program 

8. Management Operations 

DESCRIPTION OF CONTENT 

{The content of this module has not yet been specified; therefore, 
a summarization from the Design Report is provided.) 

This module will, through several episodes, introduce the student 
to selected management planning tools, such as activity analysis, flow 
charting, time and cost estimation, network planning, critical path 
analysis, and scheduling. 

FORMAT 



Several special exercises will be provided to allow the student to 
develop skills in the above activities". Self-evaluation checkpoints and 
questions win be included along with exemplary model.-; and feedback. The 
standard battery of assessment devices will also be developed 



9.0 The Educational Information Consultant 



MODULE 9.1 Orientation 



OBJECTIVES 

The trainee will be able to: 

Through the readings and instructor input, gain an understanding of 
the setting within which and Educational Information Consultant (EIC) 
works and the need for services he would provide. 

Identify alternative levels (local, regional, state, etc.) as defined 
by the readings. 

by^thl^readings^^ networks (directive, nondi recti ve, etc.) as defined 

Describe an operational information center, its purpose and function, as 
detined by the readings. 

Explain the emerging need for the Educational Information Consultant. 
DESCRIPTION OF CONTENT 

This module provides an introduction to the EIC Series with readings 
and instructor input on the material, state and regional information 
networks and their potential contribution to educational practice 
Thus, a mediated introduction to the emerging role of the EIC in which 
five basic processes are identified and briefly described. 

FORMAT 

The module is organized into three episodes or learning activities. The 
first involves reading materials and instructor discussion of various 
levels within which the EIC can function. This is followed by an audio- 
visual overview of the role of the EIC. Finally, there is a T-puzzle 
exercise to demonstrate the complexities and skills required for clear 
communication. Criteria are available for evaluating the student's 
product. 



9. The Educational Information Consultant MODULE 9.2 Negotiatio n 

and Communication 



OBJECTIVES 

The trainee will be able to: 

Name three typical modes by which an EIC can receive requests for 
information from a client: (1) face-to-face, (2) written, and (3) 
telephone • 

Observe and identify some of the conmunication techniques involved in 
receiving and negotiating a client problem via the telephone. 

Given an EIC Negotiation Checklist , record background information 
about the client and describe the general nature of the client's 
problem and the type of information needed. 

Understand the importance of using a form to record data during 
negotiation of a client problem, particularly in terms of its value 
as a kind of "contract" with the client, as a reference used sub- 
sequently in filling the client's request, and as a written record 
of the negotiation. 

Understand the importance of restating the client's problem during 
the negotiation. 

Comprehend, in terms of presenting retrieved and transformed 
information to a client, the impact of verbal and nonverbal interaction 
on the effectiveness of the communication process, as demonstrated 
in an EIC to client role-playing exercise. 

Name and define verbal and nonverbal skills necessary for effective 
communication, as outlined in The ElC/Client Communication Checklist . 

Using the ElC/Client Communication Checklist , evaluate the effectiveness 
of communication skills demonstrated in a role-playing exercise. 

Relate the communication process to the overall context of the EIC role. 
DESCRIPTION OF CONTENT 

This module contains two major learning episodes^, namely negotiation 

and communication. The negotiation episode consists of nine instructional 

elements dealing with the following activities: 

(1) Observing the Negotiation Process 

(2) Negotiating a Client Problem 



3) Designing the Negotiation Checklist 

4) Stating a Problem for Negotiation 

(5) Analyzing the Negotiation Checklist 

(6) Negotiating a Real Client Problem 

(7) Questing in the Negotiation Process 

(8) Reformulating the Real Client Problem 

(9) Synthesis of the Negotiation Process 

The Communication episode has six elements addressing the following: 

(1) Observing the Comrounication 

(2) Guidelines for Conveying Information to the Client 

(3) Preparation of a Letter Conveying Package to Client 

(4) The Written Conmunication 

(5) Application to Client Problem 

(6) Synthesis on Communication 

FORMAT 

The format of the elements include both small -and large-group 
presentations ^ role-playing exercises and evaluation discussions. 
Trainees wil; listen to taped interviews between an EIC anci a client 
Also, the student product is evaluated against criteria provided. 



9. The Educational Information Consultant MODULE 9.3 Retrieval 

and Transformation 



OBJECTIVES 

The trainee will be able to: 

Define the retrieval process and designate the skills necessary for 
effective retrieval . 

Name, in sequence, and analyze the relationships between the five 
basic operations in information searching procedure. 

Analyze the "Bibliographic Chain" and describe its relationship to 
the search strategy. 

Value the process of planning and implementing a systematic search. 

Analyze a client problem to determine the type, quantity and depth 
of information to be presented in a transformed package. Screen and 
select information items appropriate to a client problem, indicating 
the appropriate format in which to present them. 

Specify criteria used to decide on the content and format of 
information to be included in the client package. 

Become aware of the need to discriminate among alternative fonnats 
for transforming information, based on assessment of a client, nis 
problem, constraints and purpose in requesting information. 

Make judgments about assistance a client might need to use the 
package of transformed information, by indicating the need for 
"utilization aids," such as "List of Contents," "Description of 
Contents," etc. 

Value planning as an essential phase of the transformation process. 
DESCRIPTION OF CONTENT 

This module also consists of two major learning episodes. Retrieval 
and Transformation. The Retrieval episode consists of eight'elements 
dealing with: 

(1) The Search Strategy Exercise 

(2) Selecting Search Terms 

(3) Planning the Search 

(4) Beginning the Search 

(5) Subjective influences in Retrieval 



6) The ERIC Information System 

7) Reformulation and Completing the Search 
(8) Synthesis of the Retrieval 

The Transformation episode also consists of eight elements: 

(1) Introduction to the Transformation Process « 

(2) Transforming Information 

(3) Planning the Client Package 

(4) Preparing the Client Package 

(5) Selecting and Organizing Information 

(6) Summarizing Information 

7) Applying Transformation to the Client^s Problem 

8) Synthesis of the Transformation Process 

FORMAT 

These learning eleratnts provide instructor input and large group 
discussions,, readings and individual exercises^ i.>imulations using 
small teams, and slides and audiotape lessons. Criteria are provided 
for evaluation of the student's final product. 



